Collaborating with undergraduates effectively in research

by Michael Ernst (mernst@cs.washington.edu)
May, 2000

Contents:

Introduction

This document is aimed primarily at graduate students who wish to collaborate with undergraduates in their research. Also see my notes on interviewing undergraduates.

There are many benefits to collaborating with other people in your research. Many hands and minds make light work: you can accomplish more if you are working with colleagues. Having others to brainstorm with, or to keep you honest by asking questions about your ideas, is also valuable. Advising is an important skill you will need to learn at some point and that you will probably never stop learning, so getting practice is valuable. Selecting appropriate research tasks within a larger project is part of advising, and has the side benefit of forcing you to make your plans more concrete. There is great satisfaction in introducing others to the joys of research, and in helping them to grow professionally. Guiding others can help you, by showing you ways that you can improve. Above all, remember that the point of the relationship is not just to advance a specific project, but also to develop the undergraduate into an independent researcher.

Your collaboration with an undergraduate in research is not that different from your adviser's collaboration with you (a graduate student) as a researcher. How do you enjoy being treated, and what makes you effective? Duplicate these traits for your undergraduate colleagues, keep their perspective in mind, and both they and you will be happy. Don't forget, though, that different people have different preferences regarding type of tasks and amount and style of interaction.

You should avoid working with more than one or maybe two undergraduates at a time. You are still learning how to choose teammates, how to advise them, how to separate a project into subparts, etc. You can build an empire later, when you are a professor. Furthermore, you will get more benefit from working closely with a few really good, committed students than from spreading your effort across a larger pool.

Choosing an undergrad collaborator

There are two aspects to choosing an undergraduate collaborator. First is locating candidates, and second is evaluating the candidates.

Finding candidates

Every university has websites, mailing lists, and/or “job fairs” especially to match undergraduates with research projects. (For example, UW CSE has websites, the cs-ugrads mailing list, and Undergraduate Research Nights.)

If you have been a TA, you know some good undergraduates and can ask them directly. Or, ask your friends who have recently TAed. Another traditional way to find undergraduates is for a faculty member to refer them to you, after filtering for quality, interests, and working style.

You can also look up TAs for all the relevant courses and email them asking for recommendations of outstanding undergraduates who might be interested in research. Then, email those students telling them a little about your project and asking them if they had any interest. If so, I sent a list of potential projects they might work on (described in two sentences each), and set up a meeting.

When advertising for students or contacting them after getting a reference, it is very helpful to write a blurb about your project. As with all writing, get some feedback from your friends about what you have written: how does it come across? A good guide will be what you would yourself enjoy doing. (“Fix this list of 20 bugs” is not a coherent nor a compelling project. In fact, it's something that you yourself should do before the student starts, to enable him or her to proceed with a new project without unnecessary obstacles.) Make sure that you make the project sound interesting: stress imaginative aspects and eventual outcomes, even though there is inevitably a lot of less exciting programming and testing involved as well. People would rather be involved in a project where they are active participants than one in which they will be mere implementers, and they would rather do something new than merely re-implement or maintain an existing project. They are also more likely to be compelled by a project with practical and realistic impact or goals. Some undergraduates don't like projects that are too open-ended: at that stage in their career, they may prefer more direction. When you have your marketing hat on, be honest: don't promise the world in your blurb. It's unlikely that an undergrad (especially one just starting on a project) will end up with complete freedom — not even grad students have that, and they are grateful for the feedback and direction from their adviser regarding which projects and approaches are most promising.

Evaluating a candidate (interviewing)

See my notes on interviewing undergraduates.

Selecting a project

My approach is to offer a palette of options to the student. Be prepared with a list of multiple potential projects, each described in a brief paragraph, and be ready to expand on them. See which one(s) the student is most interested in. This guarantees greater interest and buyin on the project, not to mention selection of one for which the student is better qualified. It also shows respect for the student, who is not just a lackey to be ordered around. It is a gamble to say, “and if you have an idea, I would be happy to have you work on that as well.” Most undergrads don't have a good feel for what makes interesting research, nor for estimating effort, and you don't want a student to unreasonably stick to an unreasonable project. Beware of students who are equally interested in everything; that means they are truly interested in nothing. You should send them to one of your colleagues working in a different area where the student might discover his or her real love and be far more productive than with you.

There's a tradeoff between telling the student what to do and asking what he/she wants. The former may be better for undergraduates (and early-stage graduate students). I think that “choose one of these n things” is a reasonable tradeoff.

You should know (and make clear to the student) what skill set is required. This encourages ones who might otherwise have been overawed by the problem and discourages those who would not be prepared.

One good first task is to implement a standalone black box that will work with the rest of your system.

When choosing a project, be sure to know what the goal is, what is needed to accomplish it, and what you hope to learn. (It's not necessarily bad to have a desired outcome, so long as your research is honest and fair.) If the undergraduate understands the big picture, he/she will know why he/she is hacking on the code, when an alternative approach is acceptable (and when it would defeat the purpose), and why the work is important.

You should probably avoid a project on the critical path. It might be OK if you have enough else to keep you busy (that is, you really need it in three months, but you don't really need it now). If worse comes to worse, you can always just do it yourself, though that may hurt the student's ego.

Should the project be research or “just implementation”? Even you don't know what interesting problems will come up in the course of the work. It's fine to give something that seems straightforward, because you may find really interesting results. Even if the initial project is straightforward, understanding/interpreting the results may not be, and the undergraduate should be involved in that. Don't give a detailed design, just a specification of the task. Let the student do the design. Design is fun, it increases buyin and enjoyment, it teaches the student something, and you might be pleasantly surprised. Do give frequent feedback to avoid an inexperienced student getting stuck on a wrong path. Try to avoid tasks that require a lot of repetitive work that the student does not know how to automate.

Especially if you are not confident about the student's skills, you might want to first pose a well-defined, specific, small-scale starter task. This is a good way for the student to get familiarity with your project and to feel a sense of accomplishment that stems from a concrete achievement. It also allows you to assess the student quickly. The student will take much more time than you would to do the task, of course, mostly because of lack of knowledge about your toolset and codebase. One good example starter task is to install the tool, run it, and report back on the results.

Logistics

When you hire the student, settle on issues like how many hours per week the student will work. Confirm that in email. This is important to fix in both of your minds. Watch out for overcommitted undergraduates who are taking too many classes or have other jobs, etc.; you may wish to ask about this in the initial interviews.

Working over the summer can be better than during the term. You get the full attention of student. Distractions (such as classes) are the kiss of death, but 40 hours of productive work is a huge amount. It may only be feasible to start during the school year, which also helps both parties scope one another out and figure out whether a summer commitment is worthwhile. Be upfront about the possibility that it will not work out, either for you or for the student. However, have the expectation that the relationship will last for some time and be a success all around.

Some students need specific course credit, for example if they wish to complete a thesis and/or graduate with honors.

Some advisers prefer paying money to offering course credit, at least to start. It's unusual for a good student to be short on unrestricted credit (except, in some cases, for double-majors). Some students appear to take the work more seriously, and to spend more time, if it is for pay. Money requires accounting of time spent on the project (and students are used to delaying work on classes until near the deadline). Turning in a timecard is a good reminder about the student's obligations and a check on whether they are being fulfilled. Spending even a little time on the project in a steady way guarantees progress, and failure to make steady progress is the biggest potential problem in a project.

Advising

You need to accept upfront that advising a student will take enormous amounts of your time. However, people matter, and they can tell whether you care about them, so this is time both well and productively spent. For instance, you may need to teach them tools such as Emacs, a debugger, and so forth; but their work will more than reflect your investment. It's possible to advise a student using very little time; however, I don't think either party will get much out of the relationship.

At the start, it will be more work to have the student perform the task than for you to perform it yourself (though this will change over time, sometimes very quickly). On average, an undergraduate student will save you only a small amount of time compared to doing the work yourself. The variance is high, though, and there is the chance that the relationship will be a huge win. Don't view a student as a guarantee of results or as a minion, but as an investment and a gamble.

My skin crawls whenever I hear someone talking about “my student” who is “working for me”. That should be “my colleague” who is “working with me”. This is not just a matter of semantics but a fundamental key to your relationship and the respect accorded. You're likely to get what you expect. If you expect second-rate work from someone who can't make a real contribution intellectually, that will show in your attitude, and it is what you will get.

Some other advisers recommend to set a clear tone that you are the manager, not just a colleague. This means there is clear authority and hierarchy, so you get to make final decisions and the student feels worse about disappointing you.

Your adviser (a professor) may be involved in choosing the research project, but even so, the prof will not necessarily attend your weekly meeting with the undergraduate. There are some advantages to involving a professor. It lends credibility and raises the importance of the project in the undergraduate's eyes. The student may find it much easier to disappoint or blow off a graduate student than to do the same for a faculty member. The faculty member will be a better writer of recommendation letters. And, the faculty member is likely to offer good advice, especially on higher-level issues (don't burden the faculty member with low-level code details). But, all those extra weekly meetings add up to a high load on the faculty, so try to avoid them if possible. An alternative is to involve the advisor periodically, such as semi-weekly or monthly.

It's important to set a good example. Be honest in your dealings with people, in your measurements, and in your writing.

Treat each undergrad like a full group member. For example, invite them to the weekly research group meeting, give them card key access to your lab, etc.

If you care about the undergraduate's end product, then you will need to review it, give feedback, and iterate. I've had code reviews (iterated over multiple rounds of feedback) take longer than the original code took to write, but it was worth it and resulted in simpler, shorter, more robust, easier-to-understand code. The alternative would be a debugging nightmare much later, when the student had moved on. (I've had the latter experience far too often!)

Publishing

Use the same rules as for choosing the project. If you have the student write, make it just a section or two, and then edit it afterward. (Everyone is bad at a task the first few times they do it.) Some undergrads are overwhelmed by the notion of a research paper and/or view their task as restricted to coding; such students might not have any textual contributions to the paper their names are on.

Trouble

Essentially all problems in the world come from lack of communication, at least among well-meaning parties. (This is an overgeneralization, but is close to the truth.) Thus, your primary goal should be to ensure frequent communication. Meet with the undergraduate at least once a week — more during the summer. (During the summer, it's good to drop in occasionally, since you know where the student will be and when; this is a great way to keep the lines of communication open.) Frequent communication alerts both you and them when they are no longer on track. They can take action themselves. You can also take action, such as redirecting them, to working intensively with them, to helping them recognize that they don't want to work on the project any more. In order to understand the progress (and to help the undergraduate), you need to understand the approach and the implementation, at least at some level. Don't be a Dilbert manager who is ignorant of technical details.

Students should be both able to tell you what they have done (do they understand it well enough to clearly communicate it) and to show you what they have done (are there tools, documentation, or other artifacts that you can access and use?). Neither one alone is usually enough. Having both makes it clearer what has really been accomplished and keeps everyone honest. Furthermore, by reporting to you in detail (typically, both in a weekly progress report and then in the meeting), the undergrad deepens his own understanding of what he has done and may spontaneously improve upon it.

One potentially serious problem with undergraduates (and others!) is that they may believe it's OK to finish things at the last moment. That's only OK if your aim is to equal the quality of an undergraduate project that was finished at the last moment — because that is what you are guaranteed to get. Don't let this happen; communicate frequently, have milestones, and know the current status of the project.

Other problem-avoiding strategies were described elsewhere in this document, especially in the “Advising” section.

Finally, be sure to get other people's opinions besides mine. Their experiences, too, can help ensure that working with an undergraduate is rewarding for all parties.


Back to Advice compiled by Michael Ernst.

Michael Ernst