Written by Jason Eisner in 1997, for new Computer Science Ph.D. students at the
University of Pennsylvania. He was a grad student himself at the
time. Here's a Chinese translation.
How to Find Research Problems
by Jason Eisner
The biological anthropologist Loren Eiseley used to say there were two
kinds of scientists: big-bone hunters and small-bone hunters. (He
himself was a small-bone hunter, he said, fitting little bits of data
into the skeleton. If Eiseley had been a programmer, he would have
called this "bottom-up science.")
Computer science includes many different kinds of research efforts,
some of which are more tyrannosaurical than others. You can
contribute to one of these efforts in various ways.
Keep in mind: There's lots of research out there, so you have a
choice about what to work on. (Even if your advisor is very
hands-on, you still have some choice.) So, especially when you are
considering a time-consuming project, keep your long-term goals in
mind. Will it:
- About the smallest bone that you can find in Computer Science is a
reproduction or implementation of someone else's work.
While this doesn't get you points for originality, it may
be useful, both to your education and to the field. If you can make
it useful to enough people (say, by making it portable and
Web-available), it might even get your name known.
[Update: It may also be a good starting point. King
(2006) describes "how to write a publishable paper by beginning with
the replication of a published article."
More here and here.]
- A significant small bone to look for is a tweak that improves a
well-known technique. (In many
subfields, you will be expected to demonstrate objectively that your
method is an improvement.) Much research is of this kind.
When reading papers, stay on the lookout for such bones. In
particular, notice when the author may be making harmful
simplifications or arbitrary choices in his/her approach. These are
opportunities for you to try something different.
- Along the same lines, you might make a controlled comparison of two
or more algorithms, evaluating them by some objective measure of
efficiency or accuracy. Designing a clean comparison does take
thought, and carrying it out is often a lot of work.
This is usually a medium-sized bone, depending on how much work it
takes and (more important) how surprising the establishment finds
your results. Note that quantitative studies of this sort are becoming
increasingly important in some areas of CIS (e.g., operating
systems, machine learning, natural language, algorithms).
- You can thoroughly review the existing research in some area.
Note that this takes a good deal of time to do well, and is not
likely to do much for your career unless a lot of people read and
cite your lit review. (To publish you'd typically need to co-author
with a famous advisor, or else find some decent journal that is
willing to publish high-level overview articles by lowly grad
On the upside, writing a lit review will make you something of an
expert, able to talk confidently with other researchers in the
area; it will give you an idea of the shortcomings of past
research; and it may suffice for a WPE II, an M.S.E. thesis, or
the first part of a Ph.D. thesis. You can make it available to
others via your Web page or an online paper archive.
- Build a large program or device of some kind. This gets you some
name recognition, since there aren't that many big systems out
there, and it also confirms your ability as a software engineer.
However, do consider carefully: Will this system be of direct use to
anyone? If not, will it at least beat performance records? If
again not, does it have other merits, such as demonstrating how to
integrate or scale up existing techniques, or introducing a
collection of new techniques or a new perspective?
If you are only one of many participants in a lab project, be sure
that you make a "separable contribution" -- some piece of the
work that is impressive, that stands alone, and that people will
associate you with.
- Your field identifies various problems or issues as
significant. These often represent big bones in the skeleton of the
field -- problems that arise often, and whose solution makes a
difference. Get to know some of these problems and the work that's
been done on them. If you see how to achieve the first-ever solution, or
a better solution, or a different style of solution, that's a big
deal. Sometimes finding a good solution involves changing the
- If you are feeling ambitious and have a big-bone temperament,
study important papers in your branch of computer science, flip
through some conference proceedings to see what people are working
on, and ask: What problems (recognized or unrecognized) are
obstructing progress in my field? Can I solve them? If not, can
I at least formalize them? Can I prove to my colleagues that
solving them would make a difference?
- Talk to your advisor about problems that are ripe for the
plucking. Every field has its share of problems that everyone knows
are "kinda important," and that may even get mentioned a lot, but
on which no one has yet made a serious attempt. If you think you
spot such a problem, use your colleagues and the library to make
sure it hasn't been plucked yet.
- Finally, you can identify new interesting problems. This is often
not as hard as it might sound:
- Study existing (applied) systems and note what they do badly at.
- If your field is interdisciplinary, ask people in the other
discipline what they think is interesting. In fact, ask them
why they think computer scientists are irrelevant.
- In many areas, the data have a way of suggesting their own
problems. Linguists can find unexplained phenomena in any
magazine article. Systems programmers can collect data on
actual disk access patterns and study it for regularities to
exploit. Theoreticians of programming languages can look at
real programming languages, and graphics programmers can look at
real photographs and movies, for effects that they don't know
how to capture.
Finally: Now that you're in grad school and no one sets your agenda,
everything you do is open-ended. That means you can easily spend too
much time on any task you start, especially if stubborn perfectionism
or an inferiority complex leads you to feel that your work is never
good enough, or if you're subconsciously trying to put off that scary
next phase of your research.
- educate you?
- lead to even better projects?
- be an enjoyable way to spend your time?
- serve a goal that will still seem worthy 6, 12, or 48 months from now?
- be likely to "succeed" in some sense? (guideline: will it make
an interesting conference talk?)
- escape your advisor's imprecations?
- get the academic research community interested in you and your work?
- prove to an industrial employer that you have what they want?
- make you a so-called "famous grad student"?
- Don't spend eternity on background reading. Recognize that
you will have to start your work in a state of partial ignorance:
you don't have time to learn everything you need to know. That's
okay -- your professors do the same thing. In fact it's good, since
ignorance leaves your mind free to see new ways of doing things. So start
doing your own thinking early. You can alternate that with reading:
just show your ideas periodically to someone who can warn you
about related work and point you to relevant papers.
- Don't spend eternity on one problem. No solution is ever
complete. Take the time to make your work solid and beautiful and
presentable, but recognize when you've hit a point of diminishing
returns. Use project #1 to inspire project #2, which stands as
research on its own. Don't use it as the core of project #1',
#1'', etc. forever.
This page online: