Hints for Assignment 2

Thu Feb 21 20:26:22 EST 2008

You have to find all the places where the new features require a change. That includes awkgram.y for anything like until that changes the grammar, but not for adding a new built-in function like fields. Lexical analysis is in lex.c; you're adding new keywords, so they go into the table. Make sure you heed the comment at the top of the table! Built-in functions are handled by bltin() in run.c.

until statement

The AWK grammar needs new rules that specify the syntax of a until statement and creates the right kind of node in the parse tree. There is a new keyword that lexical analysis has to recognize and presumably a new function to be written and added to the program; that function defines the semantics. Find the code that handles while, then copy, paste, and edit carefully. There are two places in the grammar proper where something has to be added.

In the grammar, the components of the right hand side of a rule are referred to as $1, $2, etc. Each has a type that is set at the top of the grammar. These types keep the C compiler happy. The result of a rule is called $$. By default, $$ is set to $1, but if you want to pass some different value, you have to assign it explicitly, e.g., $$ = $3. You can just copy the code fragments for while loops and edit "while" into "until" as appropriate. Bear in mind that "while" also appears as part of do-while, and you should not touch that.

Some constants are defined in the grammar, including most of the language keywords like WHILE. Others, notably the names of built-in functions, are defined in awk.h. There is no good reason for this difference and probably never was, but it's now enshrined. This affects where you have to make changes.

Basically, AWK parses an awk program and creates a parse tree made up of Nodes in the interior and Cells at the leaves; Cells correspond to values, but they hold a variety of things, identified by type fields and often lied about with casts. This is certainly not a model of good software design. The values in Cells are usually set by setsval and setfval, and retrieved by getsval and getfval, but these ostensible interfaces are often bypassed in an effort to go faster (misplaced efficiency) or to do an operation that is more complicated.

Parse tree Nodes are created during parsing by functions named stat1(), stat2(), etc., distinguished by the number of children they have. until is really the same as while: it has two children, the statement to be repeated and the condition that terminates the loop. Some constructs of variable length use a linked list of Nodes; this includes argument lists for functions, both built-in and user-defined. In such cases, one finds the arguments by following the list. The atan2 built-in shows how to handle one optional argument; you need to handle two for fields.

During execution, AWK starts at the root of the parse tree and calls the appropriate semantic routine from run.c for each Node encountered; this is highly recursive since most Nodes have children. The function execute(Node*) is the focal point; it decides what function to call by indexing into an array of function pointers created by maketab in proctab.c and compiled into proctab.o. Return values from functions determine the results, including some control flow constructs like break that require early termination of loops.

The link between the type of syntactic object (e.g., UNTIL) stored in a node and the function that is called when that node is to be executed is set by a table. The tricky bit is that the table is defined in a C program in maketab.c, and created by compiling and running maketab to make another C file (proctab.c) that is then compiled. This is all handled by the makefile, but you have to remember to update maketab.c or your function won't get called. And if you add a new function to handle until, you will have to add its declaration to proto.h.

fields() function

There are no grammar changes but there is a new keyword, and you have to add new semantics in bltin() where the other builtin functions are handled. All of the pieces that you need for this part are in this function, but scattered and in different contexts. Find a built-in that is semantically close to fields and modify its code, using snippets from other built-ins.

AWK uses an intricate and somewhat flaky scheme of temporary Cells to hold variables, constants, and the like during computation; managing those correctly is a pain, so you should be careful to follow the existing patterns precisely or you will create a memory leak or worse. These temporary cells are returned by most semantic functions.

Somewhat the same comments apply to string management. Although there was originally a string abstraction, it's too often violated. But roughly, getsval(Cell*) returns the string stored in the Cell, but it's not your copy, so if you want to mutate it, you have to make a copy (allocating space with tostring()) and work on that. In this problem, you should not have to do this, but you will need to allocate a new string for the output. Loop over the desired fields to concatenate their values (from getsval(fieldadr(...)) into the return string.

All numbers in Awk are of type Awkfloat, which is a double. The functions getfval() and setfval() update the numeric value stored in a Cell. In bltin(), you should first call fldbld() to ensure that fields have been created. Then you can use fieldadr() to access fields: fieldadr(n) returns a Cell* pointer to the n-th field and the internal variable *NF holds the number of fields. (This is terrible exposure of information, of course.) The internal variable *FS points to the input field separator, a string that you can just use directly.

// comments

The third part is straightforward fiddling of the lexical analyzer. The lexer has a lookahead mechanism that you can use to determine what single character is coming next before actually reading it. Find the place where # comments are handled to see what the general approach is, then find some other multi-character operator to see how the lookahead is handled.