Research process

Research process enables efficient research.  It is in some way force you into doing systematic research. It normalizes the chance for experienced and less experienced students to do equally well. There are many good processes out there depending on the nature of the topics.  This is one of the typical research process.

PHASE 0: BEFORE ANYTHING ELSE - SET YOUR GOAL

    Before using this process set the target which conference you will submit and note the publication deadline.  Now schedule your time from now to your publication deadline, by estimating how much time it takes for each step below.

    For new Ph.Ds, before you go to the process below, first browse literature.  Get acquainted with the research in the past 3 years, in the conference you want to submit.  The key is just to browse quickly to get acquainted with the frontier of the research and the common methodology to do research.  Read abstracts, look at figures, how they experiment etc, so you can get a sense what is the state-of-the-art and what are the open problems. Then decide one good problem to work on.  You will likely stick to this problem for your entire period of Master/Ph.D. life.  Before you start, this short blog may help you:

    As an overview, research process can be separated into three phases: (1) identify the problem and solution (40%), (2) develop solutions and conduct experiments (30%), and (3) write, rewrite, refine (30%).

    PHASE 1: PROBLEM AND SOLUTION DEFINITION

    1. Define a problem/research question.  Define a clear, concrete, non-ambiguous problem/research question.  This is the MOST important step, if this is not well defined, your whole work and effort will go to nothing.  Please think VERY carefully and consult papers and your supervisor.  Asking a good question is one thousand times more important than finding a solution.  Trust me, it is really one thousand times or even more, or else you simply waste your time.  It is just like sailing a ship in a wrong direction.  Even the ship is running very fast and efficient, it does not really matter.  If you feel difficult defining a problem, go to Step 0 and read more papers.
       
    2. Now check literature carefully.  Now, check very carefully, what have been done already in your area relating to this problem.  What are the existing solutions?  What are their pros and cons?  Are there any remaining challenges you can address?  How your work position among this literature?   This must be done with care, do not be lazy, please, as we need to know clearly how much you have advanced the field, and also you do not want to find out later that someone has already done your research.  More how to do related work efficiently.
       
    3. Refine your problem.  Now refine your problem statement again to be more precise.  Specifically, it should clearly show the gap of past work.  Repeat 2 and 3 until you got a concrete, clear, significant problem.
       
    4. Propose a solution to the problem.  Now you are clear about your problem,  you need to propose a good solution.  Usually, it is a good practice to first check what is the current solution and what is good or bad about it.  Then come up with 3 to 4 solutions, try to do a critical analysis and try to wear the shoes of a reviewer. You should have quite some confidence that your solution will work.
       
    5. Double check the criteria for your solution.
      • Feasible:  Do you have at least a rough idea how to implement it?  How long will it take?  What is the risk that it will not work as you think?   Will you able to finish this within your given years?
      • NovelAre you sure your solution is novel?  Have you really REALLY read ALL relevant related work? (check again possible contributions) . All reviewers are SUPER knowledgeable.  Please do not use luck. Is there really no one who does this or similar ways before? 
      • Significant:  double check, are you sure your problem statement and the solution is something the research community cares about?  Why is this problem important to solve now? What is so hard about this?  Maybe it's new but there is no significance!  or Maybe you are solving something that is too easy or too obvious!  What is the "social" or "technical" significance?
      • Scientifically verifiable: Can you really show that it is a good and valid solution?  Can you prove in a solid way that the problem is solved? (there are something science cannot answer) Sometimes, to check this, it is good to think about research methodologies in advance.

        If all/some of your answer is unsure, please rethink carefully, try an alternative solution, or rescope your problem, such that your work is manageable, novel, and at the same time, significant.  You should feel quite clear and confident.

    PHASE 2:  EXPERIMENTAL DESIGN

    1. Think about your experimental design.  Now think about how you can **scientifically** prove what you claim or intend to achieve, in which metric, in which context.  This step is necessary because it makes sure you create some solution that can be scientifically proven to solve a problem.  Check the experimental design section for more tips.
       
    2. Write everything down on the paper using the format of the venues you wanted to submit (highly recommend latex template).  Now, I know you already feel itchy to start execution.  But wait, first, please write the Abstract, Introduction, Related Work, Proposed Solution, and Experimental Design, Predicted Result, and Expected Discussion section for your paper.  This is perhaps the key step that makes a paramount difference that many overlooks. This is necessary to clear your mind and force your mind to think logically and deeply.    Although it takes time, you will often (about 99.99%) to find that you found flaws in your work that you have not thought about before, without writing.   For this step, starts with an outline, answering how the work is novel, significant, and how to logically show that your solution is good and validated.  Then, based on this outline, write each section following the conventional structure.  As for beginners, please choose one model paper from good conferences where you can learn directly from.  Writing from scratch without anything to based on is very difficult and likely to get wrong.   Detailed guide here.
       
    3. Develop your solution based on your contribution type
       
    4. Run a pilot study - It is often risky and costly to right away carry out the whole experiment.  Running a cheap pilot study with two to three participants is often useful to see what happen and see whether anything about experimental design or solution needs to be changed.  You will be often surprised that there are many unexpected things happening.  Repeat this step until everything seems to be expected, where you found a trend that you will found a practical and statistically significant effect.  Typical setup guide.
       
    5. Run the experiment and analyze data.  How to do statistics?

    PHASE 3: PAPER WRITING

    1. Write and polish. Finish the rest of your paper and polish.  Remember that real writing starts when rewrite.  A good paper usually takes 2 months of rewriting (at least 15- 30 revisions, except for some geniuses, screw them!). Make good figures, good video, write concisely and clearly, it takes a lot of time and effort. Double check this criterion.
      • Novelty:  What is new here?  What can be learned from here?  And why has it not been done before?
      • Significance:  What is the social or technical significance? Why is it hard?
      • Validity:  Did you really prove what you have claimed?  
      • Clarity:  Last, did you clearly convey 1, 2, and 3?
         
    2. Prepare for rebuttal (conference) or revision (journal).  After several months, the verdicts are out.  Prepare yourself to do a good rebuttal and/or revision.  The key point for rebuttal is "Be polite. Prioritize the most important issues.  SHOW how you change, not simply what you will change."