Princeton University
Computer Science Department

Computer Science 333
Advanced Programming Techniques

Robert Dondero

Fall 2017

General Information | Topics | Assignments | Project | Schedule | Policies


The COS 333 project is an opportunity to undertake a larger and more elaborate task than any of the assignments: to design and implement a significant piece of software, working in teams of 3 to 5.

The intent is that this will not be just hacking, but a serious attempt to simulate some aspects of reality: choosing something suitable to work on, planning how to get it together, designing it before building it (though allowing for the inevitable changes of direction as things evolve), building it in stages, testing it thoroughly, trying it out on real users, and documenting and presenting the result, all as part of a small team. If you do it well, this should be something that you can show with pride to friends and prospective employers; you might even make money with it.

The project will involve many of the issues of software engineering as they occur in small, multi-person real-world projects. Some of this material will be discussed in class, and some will be found in supplementary readings.

The considerations affecting the form of the project are:

Project Definition

A large number of real-world systems are based on what is often called the three-tier model: a user interface, a database for persistent storage, and some process(ing) between them. Many web services use this architecture. For example Amazon has a web-based user interface; the data is a catalog and customer information, and the process is a wide variety of searching and data-mining operations. News and financial services are analogous: again, a user interface, a background information gathering and filing service, and mechanisms that let a client register for, access and process interesting items. Cell phone systems often have similar components, either on the device or in the cloud. Browser extensions do too.

The project this semester will be much the same as in previous semesters:

build a three-tier system for any application that appeals to you.

This is a very open-ended specification, so the big problem is likely to be defining a topic of the right size. Almost every web service will suggest something, perhaps novel or perhaps "We can do that much better"; either would likely be fine. Hiding, selecting, or merging data from existing web services is a possibility; shopping, news and other bots are examples, as are the myriad mashups that combine information from multiple sources. Cell phone applications are often like this too, as are browser extensions.

Numerous web services (including Google, Amazon, and Facebook) provide program interfaces, which might be a way to put a different face on some aspect of their systems. Using these APIs could be good for potential projects, and creating a good API is a fine example of a non-graphical user interface.

Look around campus for other possibilities: maps, tours, notification services, databases, and so on are all potentially interesting and feasible, though make sure that the information that you want to use is available -- concerns for privacy, property and turf can all get in the way of a great idea. We often hear about projects from staff, faculty and other students. You might find something appealing on this list of project ideas from other people and by talking to people in other departments. If you want to explore further, come talk to the course instructors.

Some of the best projects come from noticing a task that is done by hand or poorly by machine, when it could be done really well by a suitable program, or where something complicated could profit from a neat user interface. Think about how rapid interaction might be used in some new application, or how some standalone program could be web-ified, or how some service could work on a phone. Or maybe focus on tools that make it easier to build such things, and create a couple of examples that demonstrate the wonders of your tools.

In our experience, in successful projects the people are really turned on by what they are doing, whether because it's their own idea and that's sufficient motivation, or because they are building something that someone else cares a lot about too. Either way, that kind of engagement usually leads to the best results. Be wary of projects where none of the team is really interested, or where one member is driving the whole thing, or where the application is too far away from your own experience.

Previous projects have included web-based calendars and financial systems; numerous improvements of campus services and agencies; rental and exchange systems; course evaluation, selection and scheduling; information services for the local environment; social networks; and several map mashups.

The assignment is to create such a system, using any combination of existing tools and new code. The functionality that you must provide includes the following:

How big? There's no official requirement, but it ought to ultimately involve some thousands of lines of code of your own; churning out a thousand lines of repetitive JavaScript or mechanically generating a bunch of Python with Django isn't enough. Some systems have been as big as 10,000 lines of C++; that's more than necessary, though it was great work. In any case, we will expect to see clean, well-written code, with appropriate attention to principles of sound software engineering.

Some Options

Distributed or local? Most systems are distributed: the user interface runs on a client machine and the data is stored on a server. The processing might be at either end, or some of each. We strongly encourage systems that involve a network connection, even if it is only to a local host or the database, so as to leave open the option of distributed operation.

Languages, tools and environment? You can use any combination that appeals for any aspect -- web-based or stand-alone; Windows or Unix or Mac or phone; Java or Python or Javascript or C#; Ruby on Rails or Django; your own machine or a real server like AWS. The only restriction is that your running system must be readily accessible so you can demo it effectively and we can run it for grading. We also require the use of a version control system, preferably Git. It will save your life sometime for sure.

Make versus buy? Much modern software development is done by combining code and components that others have created. You can use as much of this as you like, as long as the finished product acknowledges the work of others, and has a substantial contribution of your own.

Things to Think About

We have attempted to make this as open-ended as possible. This is your chance to invent something new, or to do something better than others do. You might think of this as practice for a new e-business or e-service, the sort of e-thing that made some of your predecessors here (like Meg Whitman and Jeff Bezos) into e-zillionaires. Of course if you have a really good idea and are acquired and do an IPO by Dean's Date, it's an automatic A+.

But you have only about 12-13 weeks, so you can't get too carried away. Part of the assignment is to plan exactly what you are going to create, what each team member will be responsible for, and what interfaces you will require between components so independently-created pieces will fit together. What schedule will you follow? How can you work on different parts in parallel and keep them integrated? How will you ensure, if your time estimates are too optimistic (as they inevitably will be), that you have a working subset, rather than a non-working collection of partially completed pieces? How will you convince skeptical instructors that you are making progress, not just writing code (or doing nothing at all)?

Since the project will involve multiple people, a major task is to divide the work into suitable pieces, with planned interfaces. Each of these components must be a separate entity that can be implemented and tested separately. But you will have to think carefully about the interfaces between them: the problems caused by poor interfaces are a recurring theme in comments from previous projects (and, happily, so are the strongly positive experiences reported by teams that did a good job on interfaces).

The project will represent 50 percent of your course grade. All team members will get the same grade (with the potential for a correction in the unlikely event of someone not carrying their fair share of the load), so be sure that you all agree on who will do what, by when, and to what standard. Each person must contribute their fair share, including writing a reasonable fraction of the code for the system, no matter what other role they play.

Here are some things you must start thinking about now:


Since the project involves more than half a semester, it is possible to develop a significant piece of software, but serious planning and steady work will be required for your project to be completed on time. To encourage planning, organization and regular activity, the project will have several deadlines that will be enforced. [Some of the dates near the end may shift a bit; there will be adequate warning.] The TAs will be responsible for primary supervision of the teams and the lead instructor will act as backup and second-level management. You will be required to meet with your TA manager/mentor each week starting at week 5 of the semester. These meetings are a graded component of the course, so preparation and attendance are mandatory -- no excuses.

We will also try to follow good software engineering practice as much as possible. We will lean heavily on things that have been helpful in previous offerings, such as design documents, status reports and reviews aimed at helping you to organize and the instructors to monitor. All of these are also mandatory and graded.

Deliverable: Project Approval Meeting with Instructors

Your team must meet with the course's instructors for about half an hour to discuss your plans before they are set too firmly. If you need help finding a topic or choosing among several, please talk to the lead instructor and the TAs well before this meeting; we're happy to respond to ideas, make random suggestions, and help steer you, starting any time. But don't come to this meeting with 2 or 3 vague ideas, no team consensus, and no concrete plans. It is your responsibility to come up with a project and partners, so don't leave this to the last minute. If you want to explore projects that involve other parties on campus, come see the course instructors early so we can help you make connections.

Deliverable: Team Directory

Your team leader must create a Team Directory. The directory must be in the Google drive of your team's leader. The directory must be named named "333", and must be immediately subordinate to the top level of the Google drive. The team leader must share that directory with all members of your team, and with the course instructors. Throughout the semester you and your teammates must compose documents in your Team Directory. The documents that you and your teammates must compose are described below.

Deliverable: Project Overview Document

Your team must compose a Project Overview document in your Team Directory in a file named ProjectOverview. Your Project Overview must provide a thorough proposal of what you expect to accomplish in your project, and how. This must include an overview of what your system will do, the basic organization and components, a rough schedule of what you expect to accomplish each week, and who will be doing what. It must identify any significant risk factors and what you will do if bad things happen.

The purpose of the Project Overview is to encourage you to do some detailed thinking about what your project is, and how you're going to work on it. The more thinking you do ahead of time, the easier the pieces will come together when you do start to implement. Implementation before design is not a good idea.

The Project Overview must sketch out what you expect to accomplish in your project, and how. This must include an overview of what your system will do, the basic organization and components, a rough schedule of what you expect to accomplish each week, and who will be doing what. It must identify any significant risk factors and what you will do if bad things happen. ("We're expecting a large donation to buy a server; we will drop the course if this doesn't come through in time.")

The Project Overview must be roughly the equivalent of 3-7 printed pages. There's no need for fancy graphics or strong production values -- the content is what counts, though a reasonable level of detail, perhaps some diagrams and screenshots, and good English free of typos and spelling mistakes will help suggest that you have taken some care with it.

The Project Overview must contain these sections:

Section 1: Identification

Project name, project leader, team member names and email addresses.

Section 2: Elevator Speech

Your project's elevator speech must be a single paragraph that states clearly and succinctly what your system does.

The idea is this: What would you say if you were alone in an elevator with Bill Gates for 60 seconds, and had to ask him for funding for your project?

The speech should be a short attention-grabbing description without big words but with good buzzwords. You may be asked to give your team's elevator speech during one of the lectures; so be prepared.

Section 3: Overview

The Overview section must provide a general overview of what your project is, what it hopes to accomplish and the like.

Section 4: Requirements

The Requirements section must provide a few sentences describing the problem that your system will solve. More specifically, you must describe in detail the intended users of your system, how your system will benefit them, and contrast with existing solutions.

Section 5: Functionality

The Functionality section must give some typical use case scenarios. If you work through some scenarios carefully, the rest of the system will be a lot easier -- this should be done before you worry about implementation details. Joel Spolsky's articles on specifications are worth reading for a general guide.

Section 5: Design

The Design section must describe your major components. Each piece must have outlined what it does, how it connects and interacts with the other pieces, and their most likely implementation(s). At the very least you must cover your three "tiers". The business logic and/or middleware is likely "where the magic happens" with your specific project, so you will want to spell it out carefully. In this section you must pay specific attention to interfaces between components -- interfaces insulate each part of the system from details of the other parts, allowing modularity of both design and implementation. With proper interface design, you must be able to make significant changes to the details of your interface or database without changing the core code much, if at all.

Include a brief discussion of the likely implementation choices: languages and tools for various parts, operating environments for servers and clients, choice of database system, code repository, and the like. None of this is binding, but you must have thought seriously about it.

Section 6: Milestones

The Milestones section must list significant milestones that you plan to achieve every week or so. Milestones must be a concrete task or feature that is 100% done, not a vague description like "coding" or "debugging". Allow for slippage, and for all the required "overhead": demo, documentation, etc. Consider what components constitute your minimal plumbing, your prototype, your alpha and beta versions, and how you can arrange your sequence of stages so you can stop at any time and declare success.

Section 7: Risks

The Risks section is the place to show that you've thought about what you need to do and what might go wrong or cause delay. We're not interested in generic risks like someone getting sick, but in perils specific to your plan: learning new languages, tools, and systems; dependence on data or software or hardware acquisition. What will you do if your preferred path is blocked? Give this some thought, so you don't discover a month from now that you simply can't have something that you need.

Deliverable: Timeline Document

You must compose, incrementally throughout the semester, a Timeline document in your Team Directory in a file named Timeline. The Timeline must indicate, on at least a weekly basis, the tasks that each member of the team accomplished recently. You must update the Timeline throughout the second half of the course. You must review your Timeline with your TA during each weekly status meeting.

Deliverable: Weekly Status Meetings

Your team must meet with its TA each week, beginning with week 5 of the course. In the first meeting you must go over the project in more detail, describe the project schedule, the components and interfaces, and the allocation of people to tasks. It is highly desirable to plan a sequence of stages such that each represents a working system; if your schedule proves too optimistic, this gives you a fallback that you can still demonstrate.

Before each meeting with your TA, you must strive to resolve any technical issues, so that you can show concrete progress since the last meeting, ideally a perpetually working system with a monotonically increasing set of features. The meeting with the TA is a good time to discuss open issues regarding design choices, project evolution, and team responsibilities.

During meetings you must describe your progress since the previous meeting, as documented in your Timeline document in your 333 Google directory, and as demonstrable with a working version. Given the short time scale, there really has to be progress every week. The TA's will be looking for evidence that you're getting somewhere, though of course there will be setbacks and dead ends; we just want to make sure those are under control.

Our intention is that the meetings will encourage you to think about what you have done and what you're going to do next. The meetings are a chance to talk things through, and get advice and opinion from someone supportive who is looking over your shoulder. The TA's job is not to solve your technical problems, and it's certainly not to tell you what to do -- it's your project. But pay attention, especially if your TA is concerned about something.

Everyone must come to all meetings if at all possible. If you absolutely must be away, courtesy and a concern for your grade mandates sending an explanation well before the meeting. Unannounced absences, and especially total no-shows, are unacceptable.

Be prepared. You might find it a good idea to meet among yourselves the day before your TA meeting. The time you spend thinking about and organizing for your meeting will pay off. The TA's are looking for you to be obviously prepared, with an organized agenda, and volunteering information; they should not have to drag things out of you, and they will definitely be unhappy if you're clearly just winging it with no evidence of effort ahead of time.

Deliverable: Prototype

Your team must develop, and demonstrate in your weekly meeting, a bare-bones prototype that shows approximately what you are trying to do, and what your system's core components will be, and how they will be accessed. It need not do much, but it must be demonstrate end-to-end connectivity and do some minimal part of the job. You must not be considering major feature changes after this stage, though you need not have many of the features actually implemented yet.

Deliverable: Alpha Version

Your team must demonstrate in your weekly meeting an "alpha" version -- an almost working version of the core functionality of your project. "Core" means the basic operations that form the essence of your project; if you were doing Amazon 2.0, for instance, this might mean searching for books and accepting orders. "Almost working" means that wizards (you) can help experienced programmers (us) to use the system. Your code may crash and need restarting or suffer other bugs and stability issues. Ancillary features and reach goals may still be stubbed out. Your alpha version must be able to convince us that your project can be completed by the deadline.

This is the time to start getting real users to test the system.

Deliverable: Beta Version

Your team must demonstrate in your weekly meeting a "beta" version of your system. The beta version must show that all core features work, and all intended features must be installed and implemented, though not necessarily complete or stable. No major component must still be in stub form -- this is the "drop date" for features that have not been implemented. Your system at this stage must survive casual experimentation without guidance or hand-holding. It is very useful if you can get real users before the beta: ask other teams, your friends and roommates, and anyone else to try your system to see how well it works.

Drafts of written material (as described in subsequent sections of this document) must be underway.

Deliverable: Presentation and Slides

Your team must give a 20 minute public presentation of its work during reading period week. All team members must attend the presentation. You must also attend at least three other presentations, for instruction and moral support.

The presentation must include a demonstration of your work. The presentation will determine a significant portion of your grade, so you (and the course staff) want it to go as smoothly as possible. In order to assure this, start thinking about your presentation now. You could picture your presentation/demonstration as an event that will make or break your "company" -- you'll be on the spot in a foreign place, with an audience, though the audience is really just a group of very supportive potential customers, investors, friends and family. As to content, the "trade show" metaphor is useful, but you must aim for more than just a glitzy demonstration -- you must also spend at least some time discussing the architecture, the trajectory of the project, and providing evidence that you learned something.

One possible outline of your presentation would be: a short introduction of of what your system is supposed to do; a demo of a small number of stable important features; a summary of other important features; a brief overview of architecture and implementation; an anecdote of something that worked well or failed disastrously such that you were able to take something away from the experience; and something that you might explore further or do differently if you were to start all over again or had more time.

You will be allotted 20 minutes, which is not a lot of time. Plan on using about 15 minutes, to leave time for questions, setup and teardown, and other delays. In previous years teams have done well on timing; rehearse to keep the streak alive. Plan demos that do not require much typing or mousing or window-switching -- this invites things to go awry, and invites the audience to get restless while you poke around. You just don't have enough time.

Have fun. These projects are really interesting, and the enthusiasm of their presenters helps make that evident to all in attendance. Because a healthy audience amplifies that enthusiasm, each student is required to attend three other presentations and you are strongly encouraged to attend more, for edification and moral support. Visitors are welcome, so bring your friends and family. Typical projects are great; come and hear about them. We will advertise the schedule of demos in advance.

If your team composes slides to support the presentation, then you must place the slides in your Team Directory in a file named Slides.

Deliverable: User's Guide Document

Your team must compose a User's Guide document, and place it in your Team Directory in a file named UsersGuide. Assume that the reader of your User's Guide is a non-technical end user who is using your system to fulfill his/her needs.

Your User's Guide must describe what the system does and how to get it to do what it does. The User's Guide must not describe how the system works. (The Programmer's Guide must do that.)

The User's Guide must begin with a description of how to access or create/install a running version of your system. The installation part must contain enough detail that a user could install the executables on a suitable system. Think about the installation processes for some of the systems you used. There's often a file called INSTALL that explains what to do. Yours might be analogous. A single printed page might well be enough for this part.

Thereafter your User's Guide must describe how to use the system. More precisely, it must describe all features of the system in enough detail that the course instructors know of their existence, and know how to give them a try.

Even more precisely, the User's Guide must provide scenarios (alias use cases) describing how to use your system. To test your system, the course instructors will perform the instructions for each scenarios precisely as described in your User's Guide -- although sometimes with purposeful digressions.

The User's Guide is a particularly important document. The course's instructors will rely on the User's Guide to test your system. If the User's Guide is incorrect, then the instructors will have no way to know if the User's Guide indeed is incorrect or your system is flawed. In either case your grade will be penalized.

Deliverable: Programmer's Guide Document

Your team must compose a Programmer's Guide document, and place it in your Team Directory in a file named ProgrammersGuide. Assume that the reader of your Programmer's Guide is a programmer who is maintaining your system over time.

Your Programmer's Guide must describe the internal structure of your system in intermediate depth, as though for programmers who will be maintaining the code after you hand it over. Help them to understand how the system is put together, in an orderly and compact description, without getting mired in lowest-level details. The document must provide a good working description of the system and its implementation, but not a line-by-line walkthrough of the code. This also is the place where you implicitly acknowledge the software systems that you used but did not create.

The structure of your Programmer's Guide is negotiable; feel free to discuss the matter with the course instructors. Typically your Programmer's Guide must contain:

Deliverable: Product Evaluation Document

Your team must compose a Product Evaluation document in your Team Directory in a file named ProductEval. The Product Evaluation should present your evaluation of your final product.

The first part of the document must address the question "How well does the product work?" That is, does the product behave as you intended it to behave? How did you go about testing the product? Which aspects of the product are robust? Which are fragile? What are the known bugs?

The second part of the document must address the question "How well does the product meet the needs of its users?" For this section it would be appropriate to conduct a formal evaluation of your product involving "real" users. It also would be appropriate to conduct your own formal evaluation of your product using (primarily) heuristic evaluation and (secondarily) cognitive walkthrough, as described in lectures.

If your project is particularly challenging technically, then it might be unreasonable for you to perform a proper evaluation of your product within the time constraint of a single semester. In that case it might be reasonable to omit the second part of the Product Evaluation. Feel free to discuss that possibility with the course instructors.

Deliverable: Project Evaluation Document

Your team must compose a Project Evaluation document in your Team Directory in a file named ProjectEval.

The Project Evaluation should describe how the project worked out. How good was your original planning? How did your milestones go? What was your experience with design, interfaces, languages, systems, testing, etc.? What surprises, pleasant or otherwise, did you encounter on the way? What choices did you make that worked out well or badly? What would you like to do if there were more time? How would you do things differently next time?

Imagine that you are writing this section for your (friendly!) boss or professor and want to explain what you learned that could be applied to the next project. Some teams present a single unified section, synthesized from the thoughts of all team members. Other teams prefer to write separate pieces of the section, since they focused on different project components and learned different things. Either is fine, but in both cases aim for true introspection: try to avoid repetition or banal generalities like "this was really interesting and we learned a lot".

Deliverable: Peer Ratings Email

At the end of the project, you must send an email to the course's instructors that evaluates the effort and contributions of the members of your team. This is your opportunity to provide personal, confidential, reflection on the work of your teammates. Note, however, that if there are severe team harmony problems or egregious issues with a team member not pulling his/her own weight during the development of the project, you should approach your TA or instructor to seek resolution rather than waiting until the end of the semester.

You will have 3 points per person in the team, excluding yourself, to distribute across the other members of the team. (For example, for a team of 5, you will have 12 points to split among 4 people; for a team of 3 you will have 6 points to split among 2 people.) The minimal score for any person is 0.

The format of the email must be a list of netids and points allocated, followed by free-form text that explains your point allocations. Please be concise and attempt to avoid turning the email into a ranting opportunity.

Here is an example of the email for teammates Joe Student and Josephine Élève in a team of three:

jeleve 4
joestu 2

Joe contributed to the initial design of the project, but dropped off the face of the earth in April. He did get back with the program for the documentation, which I think is good, but I feel like Josephine and I really carried the project through to the demo. Josephine was a dream to work with because she was a good leader and always got her code done on time and in working order, but once Joe showed back up, she did sort of dump us with the documentation at the end.

The ratings of all other team members will be taken into account when assigning individual credit for the project. Failure to participate in the peer rating process will automatically result in a rating of 0, regardless of the ratings of your teammates.

Deliverable: Source Code

Your team must create a directory named src in your Team Directory, and must copy all source code to your src directory. That includes anything you created, especially source code and similar material. If you have help files, images, data, etc., put them in the src directory too. Sub-directories are fine. Do not include other files that are created by compilation or downloaded from somewhere else: no object files, executables, class files, Git control files, etc.

Be sure that your code is complete before you copy it to the src directory. We will be looking through your source code to get an idea of what you did and how well you did it, so everything must be there.

Deliverable: Final Product

Your final product is due at 5:00PM on the Dean's Date. For web apps, this means that we need the final URL and that the site needs to stay live for approximately two weeks beyond Dean's Date. For device apps or other stand-alone programs, this means that you will need to meet with some assortment of the course staff to do a demo on your device, where you will know that everything is compatible.


Grading for the project will be based on a number of criteria, including the quality of your:

The instructors will try to grade both the quality and quantity of your work.

You are encouraged to ask questions that will help clarify things for everyone. Murphy's Law applies to projects and their administration, so there will undoubtedly be screwups. We course instructors apologize for those in advance, but of course they too will be a simulation of reality...

This page was written mostly by Brian Kernighan and partly by Robert M. Dondero, Jr.