Princeton University
COS 217: Introduction to Programming Systems

Purpose

Background

The C preprocessor is an important part of the C programming system. Given a C source code file, the C preprocessor performs three jobs:

1. Merge "physical" lines of source code into "logical" lines. When the preprocessor detects a backslash character immediately followed by a newline character, it discards both of those characters.
2. Handle preprocessor directives (#define, #include, etc.) that reside in the source code.
3. "De-comment" (that is, remove comments from) the source code.

Certainly, the "handle preprocessor directives" job is the most difficult. But the "de-comment" job is substantial. For example, note that the C preprocessor must be sensitive to:

• The fact that a comment is a "token delimiter."  After removing a comment, the C preprocessor sometimes must insert a space in its place. 
• Line numbers. After removing a comment, the C preprocessor sometimes must insert blank lines in its place to preserve the original line numbering.
• String and character literal boundaries. The preprocessor must not consider the character sequence (/*...*/) to be a comment if it occurs inside a string literal ("...") or character literal ('...').

Your Task

Your program should be structured as a UNIX filter. That is, your program should read characters from standard input, and write characters to standard output (and possibly to standard error). Specifically, your program should read text (which presumably comprises a C program) from standard input, write that same text -- with each comment replaced by a space -- to standard output, and write error and warning messages as appropriate to standard error. A typical command-line execution of your program might look like this:

decomment < somefile.c > somefilewithoutcomments.c 2> errorandwarningmessages

The Details

Your program should:

• Replace each comment with a single space. Examples:

Standard Input	Standard Output	Standard Error
abc/*def*/ghi	abc ghi
abc/*def*/ ghi	abc ghi
abc /*def*/ghi	abc ghi

• Define "comment" as in the C89 standard. In particular, your program should consider text of the form (/* ... */) to be a comment. It should not consider text of the form (// ... ) to be a comment. Examples:

Standard Input	Standard Output	Standard Error
abc/*def*/ghi	abc ghi
abc//def	abc//def

• Allow a comment to span multiple lines. That is, your program should allow a comment to contain newline characters. Your program should not add blank lines to preserve the original line numbering. Example:

Standard Input	Standard Output	Standard Error
abc/*def ghi*/jkl mno	abc jkl mno

• Not recognize nested comments. Example:

Standard Input	Standard Output	Standard Error
abc/*def/*ghi*/jkl*/	abc jkl*/

• Handle C string literals. In particular, your program should not consider text of the form (/* ... */) that occurs within a string literal ("...") to be a comment. Examples:

Standard Input	Standard Output	Standard Error
abc"def/*ghi*/jkl"mno	abc"def/*ghi*/jkl"mno
abc/*def"ghi"jkl*/mno	abc mno
abc/*def"ghijkl*/mno	abc mno

• Handle C character literals. In particular, your program should not consider text of the form (/* ... */) that occurs within a character literal ('...') to be a comment. Examples:

Standard Input	Standard Output	Standard Error
abc'def/*ghi*/jkl'mno	abc'def/*ghi*/jkl'mno
abc/*def'ghi'jkl*/mno	abc mno
abc/*def'ghijkl*/mno	abc mno

Note that the C compiler would consider the first of those examples to be erroneous. But the C preprocessor would not, and your program should not.

• Handle escaped characters within string literals. That is, when your program reads a backslash ('\') while processing a string literal, your program should consider the next character to be an ordinary character that is devoid of any special meaning. In particular, your program should consider text of the form ("...\" ...") to be a valid string literal which happens to contain the double quote character. Examples:

Standard Input	Standard Output	Standard Error
abc"def\"ghi"jkl	abc"def\"ghi"jkl
abc"def\'ghi"jkl	abc"def\'ghi"jkl

• Handle escaped characters within character literals. That is, when your program reads a backslash ('\') while processing a character literal, your program should consider the next character to be an ordinary character that is devoid of any special meaning. In particular, your program should consider text of the form ('...\' ...') to be a valid character literal which happens to contain the quote character. Examples:

Standard Input	Standard Output	Standard Error
abc'def\'ghi'jkl	abc'def\'ghi'jkl
abc'def\"ghi'jkl	abc'def\"ghi'jkl

Note that the C compiler would consider both of those examples to be erroneous. But the C preprocessor would not, and your program should not.

• Allow multi-line C string literals, but generate warning messages when they occur. Specifically, your program should write the message "Warning: line X: newline in string literal" when a newline character occurs within a string literal. "X" should be the number of the line which contains the offending newline character. Examples:

Standard Input	Standard Output	Standard Error
abc"def ghi"jkl	abc"def ghi"jkl	Warning: line 1: newline in string literal
abc"def ghi jkl"mno	abc"def ghi jkl"mno	Warning: line 1: newline in string literal Warning: line 2: newline in string literal

• Allow multi-line C character literals, but generate warning messages when they occur. Specifically, your program should write the message "Warning: line X: newline in character literal" when a newline character occurs within a character literal. "X" should be the number of the line which contains the offending newline character. Examples:

Standard Input	Standard Output	Standard Error
abc'def ghi'jkl	abc'def ghi'jkl	Warning: line 1: newline in character literal
abc'def ghi jkl'mno	abc'def ghi jkl'mno	Warning: line 1: newline in character literal Warning: line 2: newline in character literal

Note that the C compiler would consider both of those examples to be erroneous. But the C preprocessor would not, and your program should not.

• Detect an unterminated string literal. If your program detects end-of-file before a string literal is terminated, it should write the message "Error: line X: unterminated string literal" to standard error. "X" should be the number of the line on which the unterminated string literal begins. Examples:

Standard Input	Standard Output	Standard Error
abc"def ghi jkl	abc"def ghi jkl	Warning: line 1: newline in string literal Warning: line 2: newline in string literal Warning: line 3: newline in string literal Error: line 1: unterminated string literal

• Detect an unterminated character literal. If your program detects end-of-file before a character literal is terminated, it should write the message "Error: line X: unterminated character literal" to standard error. "X" should be the number of the line on which the unterminated character literal begins. Examples:

Standard Input	Standard Output	Standard Error
abc'def ghi jkl	abc'def ghi jkl	Warning: line 1: newline in character literal Warning: line 2: newline in character literal Warning: line 3: newline in character literal Error: line 1: unterminated character literal

Note that the C compiler would consider that example to be erroneous. But the C preprocessor would not, and your program should not.

• Detect an unterminated comment. If your program detects end-of-file before a comment is terminated, it should write the message "Error: line X: unterminated comment" to standard error. "X" should be the number of the line on which the unterminated comment begins.

Standard Input	Standard Output	Standard Error
abc/*def ghi	abc	Error: line 1: unterminated comment
abc/*def ghi*	abc	Error: line 1: unterminated comment
abc/*def*ghi jkl	abc	Error: line 1: unterminated comment
abc/*def/ghi jkl	abc	Error: line 1: unterminated comment

The text that your program writes to standard output should end with a newline character.

You should not make any assumptions about the maximum length of an input line.

Suggestion: Design your program as a deterministic finite state automaton, as described in lectures.

Suggestion: You will find the standard C "getchar" function useful.

Logistics

Step 1: Create Source Code

Use xemacs to create source code in a file named decomment.c.

Limit line lengths in your source code to 78 characters. Doing so allows us to print your work in two columns, thus saving paper.

Note: For this assignment you may place all source code in the decomment.c file. You need not split your source code into multiple files. Subsequent assignments will ask you to write programs which should consist of multiple source code files.

Step 2: Preprocess, Compile, Assemble, and Link

Use the gcc command with the -Wall, -ansi, and -pedantic options to preprocess, compile, assemble, and link your program.

Step 3: Execute

Execute your program multiple times on various input files that test all logical paths through your code.

We have provided several files in hats directory /u/cos217/Assignment1. You should copy those files to your project directory, and use them to help you test your decomment program.

• sampledecomment is an executable version of a correct assignment solution. Your program should write exactly (character for character) the same data to standard output and standard error as does sampledecomment. You should test your program using commands similar to these:

sampledecomment < somefile.c > output1 2> errors1
decomment < somefile.c > output2 2> errors2
diff output1 output2
diff errors1 errors1
rm output1 errors1 output2 errors2

The UNIX "diff" command finds differences between two given files. The executions of the diff command shown above should produce no output. If the command "diff output1 output2" produces output, then sampledecomment and your program have written different characters to standard output. Similarly, if the command "diff errors1 errors2" produces output, then sampledecomment and your program have written different characters to standard error.

• Several .txt files (that is, files whose names end with ".txt") can serve as input files to your program.
• grade1, grade1diff, and grade1raw are bash scripts that will help you to test your work.

Step 4: Create a readme File

Use xemacs to create a "readme" text file that contains:

• Your name.
• A description of whatever help (if any) you received from others while doing the assignment, and the names of any individuals with whom you collaborated, as prescribed by the course "Policies" web page.
• (Optionally) An indication of how much time you spent doing the assignment.
• (Optionally) Your assessment of the assignment: Did it help you to learn? What did it help you to learn? Do you have any suggestions for improvement? Etc.
• (Optionally) Any information that will help us to grade your work in the most favorable light. In particular you should describe all known bugs.

Comments describing your code should not be in the readme file. Rather they should be integrated into the code itself.

Step 5: Submit

Submit your work electronically on hats via the command:

/u/cos217/bin/i686/submit 1 decomment.c readme

If the directory /u/cos217/bin/i686 is in your PATH environment variable, then you can abbreviate that command as:

submit 1 decomment.c readme

If you are using the bash shell and have copied files .bashrc and .bash_profile from the /u/cos217 directory to your HOME directory, then directory /u/cos217/bin/i686 indeed is in your PATH environment variable. You can examine the value of your PATH environment variable by executing the command "printenv PATH".

Grading

Program Understandability

(1) Uses a consistent and appropriate indentation scheme. All statements that are nested within a compound, if, switch, while, for, or do...while statement should be indented. Most programmers use either a 3- or 4-space indentation scheme. Note that the xemacs editor can automatically apply a consistent indentation scheme to your program.

(2) Contains descriptive identifiers. The names of variables, constants, structures, types, and functions should indicate their purpose. Remember: C can handle identifiers of any length, and the first 31 characters are significant. We encourage you to prefix each variable name with characters that indicate its type. For example, the prefix "c" might indicate that the variable is of type "char," "i" might indicate "int," "pc" might mean "pointer to char," "ui" might mean "unsigned int," etc.

(3) Contains carefully worded comments. You should begin each program file with a comment that includes your name, the number of the assignment, and the name of the file. Each function -- especially the main function -- should begin with a comment that describes what the computer does when it executes that function. That comment should explicitly state what (if anything) the computer reads from standard input (or any other stream), and what (if anything) the computer writes to standard output and standard error (or any other stream). The function's comment should also describe what the computer does when it executes that function by explicitly referring to the function's parameters and return value.