COS 333 Project: Preliminary Description (Spring 2004)

Sun Feb 1 12:54:07 EST 2004

Overview

The COS 333 project is an opportunity to work on a task larger and more elaborate than any of the assignments: design and write a significant piece of software, working in groups of 3 or 4 people.

The intent is that this will not be just hacking, but a serious attempt to simulate some aspects of reality: choosing something suitable to work on, planning how to get it together, designing it before building it (though allowing for the inevitable changes of direction as things evolve), building it in stages, testing it thoroughly, and documenting and presenting the result, all as part of a small team. If you do it well, this should be something that you can show with pride to friends and prospective employers.

The project will involve many of the issues of software engineering as they occur in small, multi-person real-world projects. Some of this material will be discussed in class, and some will be found in assigned readings.

The considerations affecting the form of the project are:

All projects have to have enough common structure to make it feasible for the instructor and TAs to manage 15 or more projects, and to grade them fairly and uniformly.
At the same time, the basic project should have plenty of room to try out interesting ideas, and freedom to use a broad spectrum of languages and tools.
It should be relevant to the general themes and topics of the course, which are programming techniques, languages, tools, and interfaces; it should not stray too far into topics better covered in other courses, like graphics, games, compilers, networks, and databases.
It should break down naturally into a few components so it can be done in teams of 3 or 4 people, and into stages so that progress can be monitored.
It should be of some intrinsic interest and potential utility.

Project Definition

A large number of real-world systems are based on what is sometimes called the "three-tier model": a user interface, a database for persistent storage, and some process(ing) between them. Many web services use this architecture. For example Amazon has a web-based user interface; the data underneath is fundamentally a large book catalog and customer information; and the process includes a wide variety of searching and retrieval operations. News and financial services are analogous: again, a user interface, a background information gathering and filing service, and mechanisms that let a client register for, access and process interesting items. Even the "build an IDE" project from COS 333 a few years ago took this form: an interface, perhaps but not necessarily web-based, for editing and control; a persistent store for recording programs and other information; and mechanisms for editing, compiling, and debugging programs.

The project this year will be essentially the same as last year:

build a 3-tier system for any application that appeals to you.

This is a very open-ended project, so the big problem is likely to be defining a suitably bite-sized topic. Almost every web service will suggest something, perhaps novel or perhaps "We can do that much better"; either would likely be fine. Hiding, selecting, or merging data from existing web services might be a possibility; shopping, news and other bots are examples. Yahoo is a good place to start; one of their core competencies seems to be to invent such special services. Google and Amazon both provide program interfaces that might be a way to put a different face on some aspect of their systems.

Look around the campus for other possibilities: online maps, tours, notification services, databases, and so on are all potentially interesting and feasible (though make sure that the information that you want to use is available -- concerns for privacy, property and turf can all get in the way of a great idea). I have been talking this up with staff and faculty friends, and there may be potential projects there if anyone wants to explore a bit.

Some of the best projects come from noticing a place where some task is done by hand or poorly by machine when it could be really well done by a suitable program, or where something complicated could profit from a neat user interface.

Previous projects have included web-based calendars and financial systems; a system that connects on-campus owners and borrowers of VHS and DVD's (Blockbuster Video plus Napster); course selection and scheduling; a chat-like information service focused on the local environment; computing the values of financial instruments. This is only a small sample, however, so give your imagination free reign.

The assignment is to create such a system, using whatever combination of existing tools and new code you like. The functionality that you must provide includes the following:

User interface: This is what the user sees: a graphical interface that supports some kind of direct interaction between user and system. Web-based interfaces are likely to be most common, but it's also fine to build something that has nothing to do with a browser.
Process: This is probably the "value added" part, since this is where you process and glue together what the user wants with information sources and repositories.
Data management: Somewhere there's some persistent data, whether maintained by your system on a local machine, or accessed as needed from the web, or synthesized on the fly. You don't have to use a database system (though I strongly encourage people to try MySQL or POSTGRES), but you do have to have some component that involves recording significant state and using it in subsequent interactions.

How big? There's no official requirement, but I think that it ought to ultimately involve some thousands of lines of code; hacking together a thousand lines of PHP somehow doesn't seem enough. Some have been as big as 10,000 lines of C++; that's more than is needed, though it was great work.

Some Options

Distributed or local? Most systems are distributed: the user interface runs on a client machine and the data is stored on a server. The processing might be at either end, or some of each. I would like to encourage systems that involve a network connection, even if it is only to a local host, so as to leave open the option of distributed operation.

Languages, tools and environment? You can use any combination that appeals for any aspect -- web-based or stand-alone; Windows or Unix or Linux or Mac; Java or VB or GTk or Javascript or C#; CGI or PHP or JSP or roll your own. The only restriction here is that your running system must be readily accessible for grading, and accessible enough that you can demo it effectively in the CS building using departmental connections to the rest of the campus.

Make versus buy? Much modern software development is done by combining components that others have created. You can do as much of this as you like, as long as the finished product acknowledges the work of others, and has enough contribution of your own.

Things to Think About

I have attempted to make this as open-ended as possible. This is your chance to invent something new, or to do something better than others do. You might think of this as practice for a new e-business or e-service, the sort of e-thing that made some of your predecessors here (like Meg Whitman and Jeff Bezos and the late Phil Goldman) into e-zillionaires. Of course if you have a really good idea and sell out to Microsoft or do an IPO by the end of the semester, it's an automatic A+.

But you have only about 10 weeks, so you can't get too carried away. Part of the assignment is to plan exactly what you are going to create, what each team member will be responsible for, and what interfaces you will require between components so independently-created pieces will fit together. What schedule will you follow? How can you work on different parts in parallel and keep them integrated? How will you ensure, if your time estimates are too optimistic (as they inevitably will be), that you have a working subset, rather than a non-working collection of partially completed pieces? How will you convince a skeptical TA and instructor that you are making progress, not just writing code?

Since the project will involve multiple people, a major task is to divide the work into suitable pieces, with planned interfaces. Each of these components must be a separate entity that can be implemented and tested separately. But you will have to think carefully about the interfaces between them: the problems caused by poor interfaces are a recurring theme in project reports (and, happily, so are the strongly positive experiences reported by groups that did a good job on interfaces). So that each person contributes equitably, it is also necessary to be explicit about the roles of each person on the team. Each person must write a reasonable fraction of the code for the system, no matter what other role they play.

The project will represent 60-70 percent of the course grade. All members of a team will get the same grade (with the potential for a correction factor in the unlikely event of someone not carrying their fair share of the load), so you must make sure that you all agree on who will do what, by when, and to what standard.

Here are some things you should start thinking about now; this list will be augmented over the next few weeks, and we will talk about it in class as well.

How big a task are you proposing to take on? It should be big enough to justify spending well over half a semester on it, but not so big that it's unrealistic.
What are the components or pieces going to be? How will you organize your project into stages so that you can stop at any point with something that works? You don't want a project where everything must be finished before anything works -- "big bang" projects are a bad idea.
What will you learn from it? It's good to try a project that will force you to learn something new, like a language or a tool or a system, but you don't want to take on too many new things all at once.
What technical issues lie in the critical path? If you plan to use some particular tool or language or component or communication technique, what quick experiments can you perform now to be sure that it works and does what you need? Connecting components across a network in the face of security restrictions is sometimes harder than might appear; you want to know about potential roadblocks early. If you need someone else's data or other resources, make sure early that you can get access to them. If you're expecting to use the beta version of something "about to be released", watch out; there's many a slip here.
How will you divide the work among the members of the team? This is partly personalities and partly interests and aptitudes. Some people are better coders, others write English better, or are great at designing web pages; some plan ahead, others work well under last-minute pressure. Some take charge naturally; others are happier with a defined task set by someone else. Try to organize yourselves to match the work to the people, and try to have a balanced group -- a team of superprogrammers can have a harder time than a team of mere mortals who work together effectively. Groups of 3 or 4 are best; I will permit groups of 5 only very reluctantly.

Schedule

The following schedule is subject to change in detail but the spirit is right. Take note.

Since the project involves more than half a semester, it is possible to develop a significant piece of software. At the same time, serious planning and steady work will be required for your project to be completed on time. To encourage planning and organization and regular activity, the project will have several deadlines that will be enforced. [Some of the dates near the end may have to be shifted a bit; there will be adequate warning.] The TAs will be responsible for primary supervision of the teams; I will act as backup and second-level management. You will be required to meet with your TA/manager about once a week after spring break. This is a graded component of the project, so attendance and preparation will matter.

We will also try to follow good software engineering practice as much as possible; in particular, this will mean using checklists and other planning forms that help organize and monitor activity. I will lean heavily on tools by Steve McConnell, plus some things that have been helpful in previous offerings.

	 S  M Tu  W Th  F  S
Feb	 1  2  3  4  5  6  7
	 8  9 10 11 12 13 14
	15 16 17 18 19 20 21
	22 23 24 25 26 27 28
	29
Mar	    1  2  3  4  5  6
	 7  8  9 10 11 12 13	initial project discussion; proposal due
	14 15 16 17 18 19 20	(spring break)
	21 22 23 24 25 26 27	detailed project plan due
	28 29 30 31
Apr	             1  2  3
	 4  5  6  7  8  9 10	project prototype
	11 12 13 14 15 16 17
	18 19 20 21 22 23 24	alpha test
	25 26 27 28 29 30	beta test
May	                   1	beta test
	 2  3  4  5  6  7  8	project presentations and demos
	 9 10 11 12 13 14 15	Dean's date; project due

March 10: Preferably well before this date, I will meet with each team for half an hour to discuss your plans before they are set too firmly. If you need help finding a topic, please talk to me and the TAs; we're happy to respond to ideas, make random suggestions, and help steer you. But it is your responsibility to come up with a project and partners, so don't leave this to the last minute.

March 12: Your team must have been formed, and you must have sketched out what you expect to accomplish in your project, and how. Each team must submit a brief (maybe 3 pages) design document for their project. This must include an overview of what your system will do, some background on how others attack the problem, the basic organization and components of your system, a rough schedule by week of what you expect to do, and who will be doing what. It should identify any significant risk factors and what you'll do if bad things happen. ("We're expecting a large donation to buy a server; we will drop the course if this doesn't come through in time.")

March 26: By this date, each group must meet with their TA to go over their project in more detail, to describe their schedule, the components and interfaces, and the allocation of people to tasks. It is highly desirable to plan a sequence of stages such that each represents a working system; if your schedule proves too optimistic, this gives you a fallback that you can still demonstrate. You must bring a draft of your design document to the meeting, and hand in a final plan by this date. Sometime in this general area, we will have meetings with 3 or 4 groups so you can discuss your own project and others in a small group.

April 9: Prototype. By this date, you should have a bare-bones prototype that shows approximately what you are trying to do, and what your system will look like. It need not do much, but it must do some minimal part of the job. You should not be considering major or even many minor feature changes after this date.

April 23: Alpha test. By this date, you must demonstrate an almost working version of the core functionality of your project. "Core" means the basic operations that form the essence of your project; if you were doing Amazon-2, for instance, this might mean searching for books and accepting orders. "Almost working" means that wizards (you) can help experienced programmers (us) to use the system. Your code may crash and need restarting. But you must be able to convince us that your project can be completed by the deadline.

April 30: Beta test. Your code should largely work, all intended features should be installed and working, no major component should be incomplete. A determined sadist might be able to break your system, but a casual experimenter should not. Drafts of written material should be underway.

May 5-6: Demo days. Each group will give a 30 minute quasi-public presentation of their work. All team members must attend the presentation. You are also strongly encouraged to attend other presentations, for instruction and moral support. A one-page marketing blurb and a web page must be available for interested users. I would also like to encourage real use by naive users: try to get your classmates on other teams, your friends and roommates, and anyone else to try your system to see how well it works.

May 11: Dean's date. Everything must be done and handed in by 5:00 PM on this date, without exception. Final submission has to include a man page, an internals document describing the implementation, and a report on how the project and the team worked out and what was learned.

Grading

Grading for the project will be based on a number of criteria, including

planning: how careful, realistic and thoughtful your plan is.
development process: how well you carried out stages of design and early implementation.
functionality: successful design and implementation of the basic task you set out to do.
implementation: clean, working code is important, and will be a significant factor in your grade.
engineering: how well your project adheres to the important ideas of the course, like sound tradeoffs, testing, appropriate efficiency, etc.
documentation: marketing blurb, web page, business plan, lessons learned: a good working description of the project and its implementation (but not a line by line walkthrough of the code).
presentation: organization, preparation, polish, interest.

There will be more information as we go along, to flesh out or clarify some of the sketchy parts here. There will also be class lectures on some of the pieces, like networking basics, user interface programming, and database software.

You are encouraged to ask questions that will help clarify things for everyone. Murphy's Law applies to projects and their administration, so there will undoubtedly be screwups. I apologize for those in advance, but of course they too will be a simulation of reality...