What are Research Projects?

For the purpose of this site, we define a research project as a project done as part of a course taken by an undergraduate or graduate student for academic credit or pay and the project typically lasts for one academic semester. In some cases, the project may be extended to multiple semesters, and/or form the Masters or Bachelors thesis for the student. A typically research project starts with the problem definition, study of existing solutions, enhancements to accomplish something new, and presenting the results in the form of a project report. This article describes the various aspects of a software research project.

Introduction

Many academic institutes in Computer Science conduct research in Networking. Networking research involves a number of members: the principal investigator(s), graduate students, external collaborators and a number of project students. Principal investigators are the ones who lay out the overall research plan and bring in funding. Faculty advisors doing research typically perform this role. Advisors guide the graduate students, both PhDs and Masters, to do the work in terms of writing software, doing measurements and publishing the results. Some projects may have external collaborators who are from another organization or university, but contribute to the project in terms of setting the directions, performing the experiments or writing the software. Typically the external collaborators are those who are funding the project or are getting the funding together with your group. Then, there are a number of students who are do parts of the project in the form of a research project.

A research project may constitute a small fraction of the overall research, but is nevertheless very cruicial. In some cases, a research project provides a first step towards evaluation or prototype of the complete software. If the first step is done in the right way, the research can progress with high confidence. On the other hand, doing the first step wrong can affect the overall future of the research. In many cases, a research project implements only a part of the overall software, say, a module or a feature. This is important because the complete software is built using such single components and single features.

Research project in real-time multimedia systems are different from other disciplines of Computer Science because of a number of reasons. Although most of the projects are software-based, some may involve interfacing with an hardware device such as X10 or RFID (Radio Frequency ID). The type of the project may encompass a variety of domains including IP routing, device driver interfacing, application level text processing and even graphical user interface development. A number of systems projects are very challenging because they compete against the top-class industrial labs and companies. When we started our Session Initiation Protocol (SIP)-based project, a number of companies such as Pingtel, Cisco, and Nokia were also developing similar products.

Types of Networking Research Project

Broadly speakly the software projects in real-time systems are of different types such as simulation, testing, performance measurement, implementing of a new protocol, building a bridge between two existing protocols, or implementing a client or server based on the protocol. For the purpose of this book we classify real-time systems projects into the following:

Development:
This involves developing a new software from scratch, or developing an independent new software module from scratch. A number of things need to be taken care of to start a new project, so that future generations do not suffer from the lack of proper software process during the initial stages. We describe various guidelines and tools to get started with a new project.
Enhancement:
This involves enhancing an existing software or module for a new feature or to perform a slightly different task than what it already does. It is extremely important to follow the existing conventions and methods for the software so that it can be easily integrated into the big picture. Usually large software have a number of utility functions and tools, that should be used consistently in the new module instead of always randomly creating new functions.
Measurement:
A number of projects are based on simulations and measurements of systems. This document is not intended for such research projects. Nevertheless, some parts of the document are useful for any research project. Measurement and simulations based projects are very crucial because not everything should be implemented completely unless one can prove that the results are expected to be good. However, we do not focus on measurement and simulations related projects.

Rest of the document focuses on software development and enhancement research projects. It also provides some information on measurement projects specifically for performance evaluation of various multimedia network applications.

Author's Background

Over the last six years, I have mentored a number of projects involving real-time systems. In addition, I have done a number of projects for research credit when I was a Masters student. The first project that I did at Columbia University was a translator between the two competing protocols for Voice-over-IP (VoIP). VoIP is an important real-time service, that allows voice calls over Internet. In 1999, there were two protocols for signaling of Internet voice calls -- the International Telecommunication Union (ITU)'s H.323 protocol suite, and the IETF's Session Initiation Protocol. A number of researchers had compared the two protocols for their relative merits. However, some forsaw they both the protocols will co-exist in near future. This gave rise to the idea of developing a translator between the two protocols so that a user of an H.323 phone such as Microsoft's Netmeeting, can communicate with another user of a SIP phone. Our initial project became a great success because it was the first such attempt to build a non-trivial translator between the two protocols, and it proved to the H.323 vendors that they were still safe even if SIP were to become popular! The result was that the SIP-H.323 research drew attention of many leading vendors of VoIP phones and gateways. Later on, the project was commercialized by a startup spin-off.

While doing the project I played around with OpenH323's software as well as Columbia University's SIP server initial implementation. To modularize the system, I created a library for common functions of H.323 and SIP, and split the specific functions into separate SIP and H.323 user agent libraries. Having a SIP user agent library separate from our SIP server implementation allowed us to further experiment with additional ideas in SIP.

The first research project I assisted with was on implementing a single line telephony gateway that allowed one to make calls to and from one's PC. The student who did the project used a Teltone Access Unit T-311 to connect a regular telephone line with a Sun workstation. An application running on the workstation, written in C++, using my SIP user agent library, translated the Teltone modem commands to SIP and vice versa. The project was a success. The project student later continued for PhD in our department and graduated before me :)

After that, I helped in two related projects on enhancing our real-time streaming server. The first one added MPEG support for video so that the server could stream video to the client, and the second one incorporated support for MP3 (MPEG-I layer 3) files in the server. Then I decided to continue at Columbia and started my PhD.

As a PhD student I started mentoring more number of students every semester, under the guidance of Prof. Schulzrinne. The research projects were mostly focussed on enhancing our Internet telephony infrastructure or developing a new module to support a new feature. Various students did research projects such as email access by telephone, interactive voice response handling, user interface design for VoIP servers, location sensing using badges, and controlling devices from phone. A number of students enhanced our test bed with features such as new audio codecs in conference server, message board in user interface, recording the conference events and media, file sharing among conference participants, and event notification and scheduling for non real-time communication.

There is no set rules or guidelines about what projects are available at any given point of time. The list typically depends on a number of factors such as what components are needed in the existing system, what new ideas are there that the advisor wants to experiment with, what new project may get external funding, and so on.

A complete list of student projects that I was associated with at Columbia University can be found at Past Projects. This document summarizes my experience and lessons learned while mentoring the many projects.

Why should you do research projects?

A question is often asked or thought about -- why should I do research projects? There is no unique answer to this, but depends on the person. In general, there are various motivations for students, mentors and faculty advisors:

For students: If you are a graduate (Masters) or undergraduate student and are interesting in doing research, the research projects are the best starting point. Research projects benefit the student in a number of ways. First, he gets a flavor of what it involve to do research. Secondly, a well done project makes sure that the student gets good recommendation letter, if he wishes to apply for work or fully funded graduate studies either in the same department or elsewhere. Thirdly, a project is good way of getting credits without writing exams or doing routine assignments. The motivation is in doing something new, and better than others! Sometimes you may get paid in addition to getting course credit. Finally, a good research project puts a shining mark on your resume.

For mentors: If you are a PhD student, you can benefit from mentoring projects students in different ways. If you do not have enough time to work on the accessories or the secondary components of your research, they can be easily delegated as research projects. Mentoring a new development makes you understand what it takes to start a new project. After mentoring several projects, one feels confident in project micro-management and this helps his personal career in future. The learning is both for the project student as well as the mentor. However, the task of the mentor becomes much more difficult in system integration if the project student does not do a good job. The key is to extract maximum quality work from the student at the same time provide him with great learning experience.

For faculty advisors: Research faculties always have tons of ideas that they would like to explore but do not have enough resources in terms of students, or finance. Providing research projects to students gives an excellent platform to get the work done. However, the challenge is two fold: pick the right project students so that there is some guarantee that the work will be done in a reasonable manner, and provide good guidance and feedback so that the student does not feel out of place or implement something completely different than what was originally conceived. If there is no suitable PhD student who can mentor the particular project then the management task may become overwhelming if not done correctly.

Logistics of a research project

A research project is not a very trivial task. A careful planning can save a lot of time and energy in future. We enumerate various tasks and formalities that one can use, to increase the potential of a successful research project.

Research project advertisement

The first thing from the student point of view is access to the Faculty advisor and his research ideas. Typically, every department has online resource listing various faculties, their research interests, current projects and potential research projects. The research project page should clearly answer the questions such as "Who can take the project?", "What research projects are available for the particular academic term?", "How to register for the project as a course?" "How the project is graded?" and any other frequently asked questions about research projects. The page should also include list of past successful projects to motivate the students. We provide answers to these questions based on our lab's research projects, so that other new faculty advisors may get started from here.

Who can take the project?

This section should clearly mention the prerequisites for taking the research project. Mention clearly if the project is a programming project and requires fluency in some programming language. A number of times the student may ambitiously take up the project only to realize later that it requires socket programming knowledge which the student does not know.

This section can read something like -- "A range of projects are available for undergraduate, masters and PhD students in Electrical Engineering and Computer Science. Candidates should have a working knowledge of at least one of the following languages: C/C++, Java, Tcl/Tk. Background in operating systems (CS4118) and/or Networks (for example, CS4119 or EE6761) is desirable. CS undergraduates must have taken CS3139 (Data Structures) and CS3156/4156 (Software Engineering) and must be fluent in C/C++ or Java." Also mention any restrictions as required by the department or the school. For example, our department has three levels of research project courses: 3000, 4000 and 6000. Our restrictions says, "CS undergraduates need to know that 3000-level projects must be completed first and may be taken for only up to 3 credits. 4000 level follows which also may be taken for no more than 3 credits. It is rare for an undergrad to register for the 6000 level, but this should take place only after having completed 3 points of each 3000 and 4000-level projects." Such clarifications avoid any headaches later, for both the Faculty and the student, in case the student registers in the wrong course.

What projects are available?

This section of the web page should link to various research projects that are available. The projects may be classified based on the type such as software, hardware, or measurement; or on the skill set such as C/C++, Java, device drivers, and so on. List may continue with editing from one term to another, based what projects are completed and what new projects are desired.

The list of projects should contain a title and a brief description of every project. The description should not be too verbose, but should cover all important aspects of the project requirement in a few lines. You also mention the required skills for the particular project in terms of programming languages, Operating Systems and tools, if any. If the project is assigned, the name and email address of the student doing the project can be included along with the description.

An example description for a project titled "Screen sharing" reads as, "Building on an earlier project, enhance vnc, a cross-platform screen sharing tool so that it generates an RTP stream of PNG 'tiles' whenever segments of the screen change. These tiles can be directed to either a multicast or unicast address. A receiver application should be built that can displays these results. Language: C/C++ and Tcl. Platform: Windows or Unix." The keywords such as "vnc", "RTP", and "PNG" may be linked to the corresponding websites describing them

Another example description for a project named "Columbia Audio Tool" is as follows: "Using our existing audio and RTP libraries, build a robust RTP/RTCP audio tool that supports low delay audio using adaptive playout mechanism. Integrate forward error correction support. Modify the existing audio libraries to shared objects (.so or .dll) that can be loaded and unloaded as and when needed. Integrate the existing G.711, GSM, DVI and G.722 libraries. The tool should work on Windows, Linux, and Solaris. If possible, also on new Mac machine. Test the tool using our SIP user agents: sipc and sipua. Your tools should also expose a simple Tcl interface in C, such that one can build Tcl based GUI. The user interface should allow construction of a volume meter (VU meter), for example. Language: C. (Parts of this project may also be taken for credit)"

How to register for the project?

In this section you can copy the text from registrar's web site about the possible course registration information for research projects. In our lab, students can register research project as courses numbers COMS E6901, W3998 and W4901 under Section 008, and ELEN 6001/6002, 9001/9002 under Section 016. Mention which courses can be counted towards which degree, if such restrictions apply. Mention the level of each course, such as undergrad, graduate or advanced graduate. Also indicate the time commitment for each course credit. In general, a 3-credit course has 9 hours/week of time commitment, and many courses allow flexible credits.

How am I graded?

This is the most important question for a student taking the research project for course credit. It should be made clear in advance, regarding the requirements for a successful project. Although, there are no fixed criteria for evaluating the project, some general guidelines apply to decide the success level of the project. Moreover, if a number of students do projects on similar topics or in similar research area, the advisor can get a general idea of what level the completed project is.

Since the research project needs to be finished within the limited time frame of a semester, the student may not get enough time to accomodate all the phases of software engineering in the project. Moreover, some of the requirements of the project is not clear due to the nature of research, the software development usually happens in stages or phases.

A software programming project can have certain requirements to get a good grade. Some of the requirements are listed below:

  • Submit a initial project plan document with the help of your mentor. The plan should clearly indicate what software are you extending? what new features you want to support with priority of each? what existing libraries and modules you plan to use? what programming languages and tools you will be using? which operating systems do you plan to support? what GUI toolkit if any will you be using? how will the system configure itself? How is error logging to be implemented if applicable? what will be the installation mechanism, or whether it will be integrated into an existing project? The plan should also list major milestones, and indicate how much will be complete for mid-term review.
  • The code must confirm to the project coding guidelines. Every large project evolves its own coding guidelines. Having a non-conformant piece in the software is very annoying to future developers and maintainers. The project mentor should make sure that the submitted code confirms to the coding guidelines, before integrating it into the big system.
  • All projects typically require a documentation. We will describe the documentation in detail in the later part of the document. In general, the documentation should include the overall project report, a man-page style document for new programs, and the API documentation generated using Javadoc, doc++ or similar from the code.
  • Code submission can be done in one of the two ways: for enhancing an existing project, the code should be integrated into the existing system with the approval and help of the project mentor; and for new projects or modules, the code should be packaged using standard Unix style tar-gz with README and INSTALL files. A student who developed a new program may be asked to unpack, install and show the results during the final demonstration.
  • A final demonstration accompanied by presentation slides if needed, of the full or the partial functionality of the project is required to get a grade.

In addition to this some advisors may enforce mid-term review and bi-weekly progress report to ensure consistent progress.

The final grade of the student depends of all the factors: successful demonstration, project documentation, software code quality, consistency of the progress. The grade is assigned by the Faculty advisor, but the project mentor is usually consulted to evaluate the quality of the software.

Sometimes the student is unable to finish the project by the end of the semester. In rare cases, extensions can be granted, but only after the advisor and the mentor feel that the student has made significant progress and can successfully finish the project if given additional time.

Frequently asked question

Other than how the student is graded, the project page should also answer any frequently asked questions regarding research projects such as the following:

Who supervises projects?
The faculty advisor is responsible for the overall project direction and evaluation. Usually, a PhD or senior MS student is assigned as a mentor to assist with the details and provide hands-on advice. A project student is expected to meet regularly with their mentor and report back, by email, on their progress in biweekly intervals (every other Friday) to both their mentor and the advisor.
What is the time commitment?
The workload is designed to be equivalent to a class. Given a 15-credit load, each credit hour should consume about 3 hours, so that roughly 9 hours of work a week is expected for a 3-credit project.
Do I have to register for credits?
Not necessarily. You can also do the project in one semester and get credit in a following semester. The opposite is not possible... (Many students choose this deferred credit during the summer, to avoid paying summer tuition.)
Is there pay?
Under exceptional circumstances. Generally, the project needs to be part of a funded research project and the student needs to have shown exceptional ability, e.g., in another project under faculty's supervision or in a class that the faculty taught.

Choosing a research project

Once the student decides on taking a research project in networking systems, he needs to decide which project should he take? Unlike a standard course structure, research projects do not have a well defined structure before hand. Every research project is unique and may involve efforts quite different from some other research project.

The student has to carefully and subjectively measure the relative importance of what is interesting to him, what is helpful to him for his job, future courses or career, how much will he learn new skills against how much will he use his skills, and what is do-able given his skill sets. Many times students get too ambitious and take a project that is much more challenging that what they expected. This results in incomplete project and in the end frustation to the student and the mentor, and disappointment to the advisor. On the other hand, a successful project, even if not too challenging initially, boosts the students confidence to take up more projects, and builds his reputation with the advisor and the mentor. If you are doing the project in the hope of getting financial support in subsequent semesters, you must be very careful in choosing the project.

In this section we try to provide guidelines to the students in choosing the right project.

What is your programming level?

Every individual has a unique set of skills. You may not be familiar with C++ but you may know network simulations inside out. You have not be a fast programmer but you may be good in writing solid code. The Internet protocol may amuse you but you may find graphical user interface design as your left hand's job. Whatever your skills may be, it is extremely important that you know how good or how bad you are. Once you know your level you can harness your strengths and work hard on your weaknesses. No one is perfect, but you need to get the right balance to excel in your area of interest.

Since we are dealing with software programming projects, the first thing you need to check is the programming languages and your fluency. Most good programmers are fluent in multiple programming languages. If you know only C, then it is high time to learn C++, Python, Java, Perl or some other. Once you are fluent in one programming language, it is relatively easy to learn another one. The learning of a language happens in two steps: first you learn the syntax, then you learn the language. If you know C, you can easily learn the C++ syntax. But to learn the language and to think in C++ or object oriented paradigm will happen only after you have implemented some significant projects in C++. Think of this as equivalent of learning a new non-native natural language such as French or Spanish. When you start as a beginner, you start with alphabets and words, forming small sentences, and paying special attention to the format and grammar. Once you become fluent, the language comes naturally to you, and you can think in that language without having to translate to and from your native language. Similarly, once you learn C++, you can automatically start thinking in C++ instead of translating to and from C. Knowing the syntax and semantics is science, knowing the language is an art. If you know the art of programming, you can easily learn the science behind a new programming language and become fluent in that.

There are some tests that can help you get started with knowing your level in C and C++. These tests are only meant as an informal way only for your knowledge. Others do not care what score you get in the test or whether you know about little endian and big endian concepts. Some people may be good at knowing the syntax and semantics of the language, but may not be capable of writing even a small independent piece of code such as reversing a linked list. If you belong to this category, then you should build your programming skills first by doing some class programming assignments before taking up a research project in systems. Alternatively, you can consider taking lower credits for the research project if the longest software project you did earlier was, say, 200 lines of C code. If, however, you have experience with large Java programs, but are ready to learn C++, we strongly suggest that you consider taking the research project. As mentioned earlier, once you know the art of programming languages, it is easier to learn new languages.

Are you motivated for research?

A programming project needs someone who has the motivation to learn how to write large software. We provide guidelines to assist such students, but the condition is that the student should be enthusiastic enough to follow those guidelines to meet the expectations of a good quality research project.

A research project is not only software development, but also research. The motivation is not only for writing software, but also for learning new technology and exploring new research directions. Most of the systems research are focused around building a system that comprehensively implements and evaluates a new technology. There is scope to learn a lot since you actually work on the problem and solve any unexpected obstacles rather than reading the solutions in a book. Unlike other disciplines where one can prove a single theorem or create a new mathematical formula, systems research usually comprises of bits and pieces of small components. These small components together form a big system that provides a comprehensive solution to a research or engineering problem. The problems occur in real-time and you need to find a solution by any means -- read references on the technology, search on the Internet, ask on related mailing-lists, discuss with the advisor and mentor, or occassionally, come up with a brilliant innovative solution that no one has done before! This is systems research, generally speaking.

Usually the work load in a research project is comparable to what you would do in a course work with same number of credits. If your reason of taking research project is that you can avoid all the assignments of a course work, then you should give it another thought. Your project advisor will not award you with a good grade if you can not meet the expectations of the research project. In fact, a research project sometimes requires more work than a regular course work. I can say this based on my experience mentoring various past projects and seeing some of the students working in the lab late nights to finish the project. The advantage of a research project is that you get awarded generously in terms of grade and recommendation if you show good work.

What is your area of interest?

Most of the students who get admitted to undergrad programme, usually do not have a very specific area of interest. The interest develops during the course of study while experimenting with different areas as part of course work or research project. Usually, the students who are admitted to Masters programme have some idea of what they would like to concentrate on.

If you do not already have an area of interest, this is the right time to start exploring various areas. You can talk to various people in different research groups, including faculty and students. You should ask them about background reading material. To learn more about any area, you have to read a lot about the earlier work done in that area.

Is the project helpful to me?

Usually there are two main goals of a research project: to get something done, and to improve the skills of the students. There is a tradeoff between the two. If you already know everything to do the project, you can get things done really quickly, but will not be able to learn anything new. If you do not know enough, then you will spend most of the time doing background work and learning new skills, without much contribution to the actual project. If you do a project that does not help you in improving your skills, either in programming, performance measurement or understanding of a new system, the project is not much helpful to you.

Learning new skills and technologies are always helpful either in near future or in long term career. Specific skills such as a new programming language or use of a new Unix tool can also help you in other graduate courses.

How much will I learn?

The next obvious question is how much you will learn as part of the project. Most of the time we prefer the students who already know the programming language the project is going to involve. However, the project itself involves a number of new technologies and protocols, that the students find the project new and challenging. In the course of doing the project, the student learns many new skills such as using programming and debugging tools, how to do performance measurement, how to parse protocol messages, and so on. Moreover, the students can also develop the art of programming including the coding convention, software engineering concepts and ability to think object-oriented versus event-driven systems.

How much is do-able?

One of the ways in which a research project is different from a class assignment is that a research project may not be as well defined. This is because the eventual goal, or what can be achieved in time during the project may not be known when the project idea was conceived.

Usually this can be negotiated with the mentor and the advisor during the project, based on the skill set of the student and the project complexity. Sometimes the project is extended to the subsequent semester to complete the remaining parts of the project that were not planned earlier. Depending on the project and extensions, this may be credited as a new project or just a completion of the old semester's project for grading purpose.

During the project

In this section, we summarize various steps that a student and his mentor does during the project. Usually a research project is done in a incremental way: do the first step, then based on the outcome do the next, and so on. In particular, the software life cycle of waterfall model is rarely implemented. However, there are still some steps that are common and needs to be performed. For example, the initial planning and reading phase, core software development and evaluation phase, basic testing for demo-able outcome, and final documentation and demonstration.

Planning and reading phase

Once you have decided on the project you need to plan with the mentor. Make a web page for your project. A project web page is what others can see your project through. You may have to learn HTML to write web pages. If you use myprojectguide.org, you can use our ready-made templates to make your project site.

The content of the project page varies from project to project. However, you can put general things such as various milestones, checklists, current progress, any reading material, slides and documentation. If the software is open source or free, you can even put the evaluation version on the page. You can even put excerpt from important email discussion with your mentor or advisor, or other concerned people on the project.

The project page is a way for the student and mentor to keep track of the progress, and for the student to keep all the ideas together instead of scattered in multiple emails or hidden in their brain noodles. This is very helpful in writing final project report or future technical report on the project.

The project plan should include details on weekly or bi-weekly meeting schedule, and other milestones such as interim reports and mid-term reviews, if any. At the end of each meeting you can update the project page with the meeting minutes for that meeting. I find references to other work very helpful on the project page.

During the initial part of the project, you will need to understand the various technologies and protocols used in the project. This involves a lot of reading. For example, if you have to develop a streaming client, you may want to read on streaming control protocol (RTSP), media transport protocol for real-time communication (RTP) and information on any existing streaming client and server. It does not harm if you read more than what is required or suggested by the mentor. In particular, if you do not understand a particular concept in a paper, you can always refer to the references and read them in turn. Then, you will need to understand the project test bed, in particular, the compilation tools and build setup. We provide introduction to some of the commonly used tools in this book.

Software development phase

As mentioned earlier, the software development phase is incremental. Some parts of the project may get changed or may get added or deleted from the project requirement. You may need to learn new technology in the middle of the project to enhance it, for example. Thus, the project micro-planning and software development phases can go hand-in-hand.

You should strive to meet your weekly and bi-weekly deadlines. However, you do not need to do micro-planning for more than two weeks at a time. For example, you can set a target that "this week you will implement so-and-so file with so-and-so features." However, it is important to review your weekly progress in light of the overall project goal and discuss it with your mentor regularly.

We give various guidelines and information of tools and skills to help you during the software development phase. In particular the software development tools and coding convention form an important part of software development. Secondly, reuse of existing libraries can speed the development and reduce the chances of introducing mistakes and bugs in your code.

Some testing and review is required

When you write code, you should always think critically: where can this function go wrong? what can make this function behave in an unexpected manner? how can I write the function such that it gracefully handles unexpected input? This defensive programming is one aspect of the art of good programming. The other is testing.

Software testing comes at different scales: unit testing that deals with individual function or file at at time, integration testing that tests the interfaces to various modules so that they can be integrated, and system testing that can test the whole system as a black box, for example, to find out whether it meets the requirements or not. The cost of testing increases as the size grows, thus unit testing is easy to perform whereas system testing requires dedicated and separate testers whose job is to break the system.

Due to various restrictions in terms of duration of a research project, and the limited resources (people) working on the project as well as the nature of the incremental software development, it is difficult to employ all the types of testing in a research project. At the least the software is expected to perform good with the given good set of input during the final demonstration. However, there is usually no formal test suite to test the system for bad input in a research project.

However, a software bug can always manifest itself unexpectedly. It may happen and most likely will happen during your final demonstration to your advisor and may nulify the hours of hard work you put in finishing the project. (Remember the crash of Windows XP when Bill Gates was demonstrating the launch!) Thus, you should always put effort to catch software bugs as early as possible.

Unit testing is usually the best way to catch the bugs as early as possible. For example, you write a function to parse protocol message. You can easily write a small test routine to test this function whether it behaves right when the packet exceeds the expected size, when the message is malformed with illegal characters, and so on. Python programming language facilitates easy unit testing by integrating the tests in the document itself. Due to time contraint, you should also be careful not to over-do this.

Another way of catching the software bugs early is to do code review with your mentor. During the review you should emphasize on coding convention and reuse of existing libraries, rather than re-implementing similar functions. In particular, for application code in C++ you should use standard template libraries, instead of implementing your own dictionary or linked list. A number of small reusable libraries and functions can help and speed new software development.

Writing documents

Project documentation is probably the most important part of a research project. Many people who do not have access to your software or do not have time to evaluate and run your software, will rely and trust your results from your project report. Future project students who are using your project will appreciate your contribution if you took time to formally document the software API of your code.

There are different types of documents that are desired in a research project and depends on the particular project. You should discuss with your mentor to decide on what documentation you are going to provide at the end of your project. In particular, a detailed project report is required that describes what the project is about and what and how it achieved the desired results. If you developed a new software tool, you can provide a user manual for the software either using Unix man page style documentation or some other easy to use format. The purpose of a user manual is to assist a user, typically non technical and someone who doesn't know the details of your project or the technology used behind it, to use your software. Besides the project report and user manual, you can also create a API documentation generated from code comments using tools such as docify or doc++ to help future developers to understand and extend your software without going through the source code.

The project report can be written in various styles. In a tutorial style report format, you provide a tutorial on how your exactly did the project. For example, if you extended a conference server to add a new audio codec, you can provide guidelines on how to extend the conference server for new codec so that someone else can further extend it with another codec. If such a tutorial already exists then you do not need to repeat the details but just provide enough information as an appendix to the previous tutorial. Another style is a research paper style, in which you motivate the reader on why the project is important and describe what results you obtained, without necessarily giving details of how your implemented your software. Both the tutorial and paper styles are helpful and which style you follow depends on your particular project. We suggest including details for both, so that it gives all the necessary information for another student to repeat your project, it gives enough information to do a similar project or extension to your project, and it gives good information to get the gist of why your project is important and what important results it achieves.

Final demonstration

The final demonstration of your research project to your advisor is very important in the project evaluation and grading. Usually, the demo happens in the final exams week or the week before that. Since the grades needs to be turned in within two days after the last final exam, any delay beyond the finals week means that you will get an incomplete grade now, and your grade will be changed later. Thus, if you are planning to graduate this semester, you must plan ahead to finish all components of your evaluation including the demo during or before the final exams week.

If there are lots of project students for an advisor, he may prefer to schedule all the project demos on the same day for ease of scheduling. You should talk to your mentor and advisor to schedule the demo at least two weeks before your final exams week. You should also make sure that your mentor is available during the demo schedule so that he can provide more comments or feedback during the demo, or may even help you answer some of the questions that your advisor might have but you do not know the answer.

In any case, you should note that you will usually get only one chance to demonstrate your project. Thus, if it fails during the demonstration, either you need to provide a valid explanation and/or the project may be termed as incomplete and your grade may be affected.

The duration of the final demo may vary depending on your project. We usually allocate half hour per student. The demo includes a brief presentation to describe what the student did, followed by the actual demo of the software project, and at the end questions and feedback by the advisor.

The presentation should be as brief and concise as possible, without giving much details about the motivation, but focussing more on what you actually did and what is the final outcome. For software development project, the presentation can be say less than five minutes with one or two slides describing the bulleted points of what you did. For performance evaluation projects, the presentation may be longer including the graphs and results of your evaluation.

The actual demo is when you show in real-time how your system works. If there is some background work needed to setup the system, such as running various servers and support tools on various machines, you should make sure that they are up and running before your start your demo. Thus, the actual demo should only walk through the main features of the system without going through the installation and setup of the system, unless the project itself concerns "new installation setup for VoIP", for example.

The advisor may interfere during the demo to ask questions, usually to clarify how it works. He may also try to test the system for basic checks. For example, if your have a user interface element that expects a phone number, he may enter a text string to see if the system gracefully handles invalid input. But most of the time, the advisor will believe you and your mentor in what you say about how it works. Thus, it is the job of the mentor to clarify in case of any misunderstanding between the advisor and the student about the working of the system. The advisor has rough idea of how the system works, whereas the mentor has more clear picture of the specific system based on the regular meetings with the student, as well as the big picture of the overall system. In many cases the student is in fact extending the system that the mentor worked on earlier.

The questions asked by the advisor are typically of two types: clarification and extension. The clarification questions are to understand how exactly the system works and what exactly the student did. The extension questions are typically to understand and apply the project to other projects, to extend it for new work, and to see how it fits together in the big picture.

Usually the advisor provides feedback on the system during the demo, on how to extend it and what are limitations. The student is expected to update his project report with such details, and in some cases may even want to extend his software to incorporate some comments. For example, fixing the user interface to display date in a different format, may well be done in this project; but adding support for a new protocol to interwork in a streaming client may form a future work item for the project.

Ideally, you should be ready with your project report and software package, before the demo. However, you may need to resubmit your report and software if some changes are needed. In any case, you must submit all deliverables in time to get a grade, as mentioned earlier.

Final submission

Your final set of deliverables includes all the source code, documents and results for the project. All documents should be sent to both the advisor and mentor. The source code can be either sent to both the advisor and mentor, or checked in to the source repository of the project. Instead of submitting individual files as email attachments, you should submit a single tar'ed archive of all the documents including any figures.

There is usually no size restriction for the project report. Secondly, a web page report can be split across multiple web pages. In our group, we do not accept Microsoft Word-only documents, but allow PDF and HTML formats. If you submit a PDF document, then you should also submit the corresponding source files such as Latex and Word document, along with any diagrams you used (e.g., xfig files).

Grading

A number of components form the project evaluation criteria, as described earlier. We include the following criteria for evaluating a project.

Effort:
How much effort did the project involve? How much effort did the student put in to the project? Is the software already integrated in to the overall system? This usually includes feedback from the mentor, e.g., "the student implemented various auto-attendant features using vxml.tcl library and our SQL tables. For the two-point project, the student did a lot of work." Thus, indicating what the project involved and what the student did.
Quality:
Did the software follow coding style? Does the system handle corner cases and graceful handle failures? Is the software in demo quality (with lots of printf and commented unused code) or a good product quality (with comments and description on various functions and classes)? Usually the mentor provides the evaluation after looking at the submitted code.
Hand-holding:
Was the student an independent thinker and worker or did he require low-level spoon feeding during the project? How much assistance did the mentor provide in actual software development versus the discussion of the project? Was the student able to grasp and learn new technologies and skills easily? This does not directly affect the grade, but is useful in future prospects of projects and further graduate studies under the supervision of the advisor. Moreover, this also helps in any recommendation letter that the advisor or mentor may need to write for the student.
Programming skills:
How good the student is as a programmer? Is he capable of building large systems? Is he capable of contributing significantly to existing projects? This again does not decides the grades, but is useful in future prospects of the student in working with the advisor and on this project or others projects.
Continous progress:
A continous progress is essential to make sure that enough effort are put in completing the project, instead of just finishing a bare-necessary features for a demo during the last few days. In particular, the bi-weekly reports to the advisor and mentor, and the mid-term review are considered when evaluating the project. If the student was able to finish the project's essentials much before the end of the semester, he would have had enough time to ponder on the completeness and quality of the project and thus would have made a better project, compared to the one when the student finished 90% of the work in the last two days before the demo.
Demo:
This is the final outcome of what features were implemented and demonstrated at the end of the project. The final demo is the time when the advisor gets the impression of the completeness of the project and general high level quality and effort put by the student in the project.
Documents:
A good and completed project report is necessary to get a good grade. Besides the project report, other forms of documents are also helpful and positively affect the grade. For example, an API documentation of a new library or an extended user manual of a new software tool, gives an impression that the student put a lot of effort in successfully completing the project, instead of doing the work in a haste during the last few weeks or days before the demo.

Again, there is no strict objective guidelines on grading for a research project. The decision is made subjectively by the advisor, upon feedback from the mentor, based on various factors: effort, quality, continous progress, successful demo, and completed documents. The advisor compares the performance of the project with the other projects done with him either in that semester or past several semesters, to evaluate the comparative strengths and weaknesses before assigning the grade. In my experience, a successful project most likely gets an A. If the student missed on certain points such as failed to submit bi-weekly report (even though the progress was continous), then the grade may drop to A-. An unsuccessful project because of the lack of effort or continous progress by the student is usually heavily penalized. Sometimes, the student may be given another chance to complete the project in the subsequent semester.

After the project

A successful project builds a good reputation of the student with the advisor and the mentor. This is helpful if you are planning to apply for graduate studies or work, or elsewhere and need a recommendation letter from the advisor, or in some cases the mentor. Most of the graduate research assistants in our group were appointed based on their performance in a research project. Thus, a research project is also an evaluation opportunity for the advisor to discover good students for graduate studies, and an opportunity for the students to get discovered for funding of graduate studies!

Some students continue in a related project in the subsequent semesters. Alternatively, some have extended the same project for another new feature or worked on a completely new project with the same advisor.

A research project is done for research. Thus, the student should also try to publish his work in a scholarly conference or workshop to invite feedback from other people in the community. As a short-term goal, the student should prepare a technical report with the help of his mentor and advisor to document his research contributions. A truely novel idea may also be filed for patent after discussing with the advisor. However, any publication of the idea before the patent application is filed nulifies the patent.