Sunday, June 9, 2013

4b. Reviewing/Writing guidelines: Methodology.

An adequate process for reviewing papers comes from the classical teaching/ learning methodology, which is based on the tracking routine:

  1. Expose the idea. As the teacher gives a class, the author illustrates the proposal's hypothesis. It creates the Introduction section. This idea pretend to solve a well--defined problem. So, the problem should be adequately illustrated as well.
  2. Give an example. The example is the keynote in a class session, since the student knows how the problem is solved until now. It provides the ultimate way to solve the problem proposed in literature and state-of-the art. So, the author learns about the current methodologies, techniques, and experiments to be overcome by the proposed one.
  3. Expose the experimental setup. It means the questions of the exam. Here, the author should probe that he has the enough knowledge to carry the experiment and knows that results will be remarkable. Comparing with a classroom's exam, where the student should prove that he knows about a matter or topic in particular, and has the skills to solve a related problem.
  4. Make the experiment and expose the results. At the classroom, the student answers the stated questions. Accordingly, the results are the answers which indicate the solution of the problem. So, scoring of those results indicates if the experimental setup was accomplished, the author achieved the sufficient knowledge to solve the problem, and the impact (poor/slightly/good/remarkable).
  5. Evaluation by pairs. It is the part where the teacher scores your exam. As the teacher scores the items of exam/presentation/poster, from two to four people provide the advances/failures that the proposal provides. Due to the relevance of a scientific publication, and to provide fairness, the number of evaluators is more than one. In contrast, the impact of the school exam covers the particular learning aspect of the student. So, a single evaluator can complete this task. However, the fairness is improved when an invigilator - a person different from teacher - monitories the school exam. In the case of a scientific publication, an editing committee meet the functions of invigilator, making sure that the reviewing process goes clean and fair. 
  6. Discussion, rebuttal and fixture. Aside of some impairment from the pairs, which is a theme that the committee should concern, the scorings of the proposal give an indication of the impact, appropriateness and novelty of the proposal in the publication/conference/dissertation where the proposal takes place. Here is where the discussion section of document is critical. This is the section where the most of the commentaries, annotations, and corrections would be. In the case of the school exam, the teacher can easily provide a score by checking the good/wrong answers, and can weight those results by relevance (easy/tough exercise/question). Besides, the evaluation by pairs takes a slightly more deep process. There are various possibilities of scoring ranking and its correspondent actions :
    • If the corrections spread along the document, specially in orthographic, semantics, language usage, and document structural errors, the real impact of the proposal hides. So, it is important to first correct this errors, to avoid stressful reviewing/revision processes.    
    • Else if the corrections relate to incomplete/not-properly-commented references in the introduction/overview/background section, the author knows that a literature revision should be done again.
    • Else if corrections are around the experimental-setup and results, author should correct the methodology, and rearrange the floats (tables, figures) to provide a re-interpretation of results. A too-long table (many row/columns, many subdivisions or over-paged) indicates that the proposal was unfocused, because it evidences that the proposal pretends to adjust many parameters at the same time. A large number of poor--commented figures, which lead to documents with only-figure pages, or figures stacked at the end of the paper, indicates that the experimental setup was not properly deployed. An image is worth a thousand words, but a thousand words should give a worthy explanation on the results that image provides. It is like the critical interpretation of artwork (painting, sculpture, cuisine, music performance, reliquary). 
    • Finally, when most of corrections come together into the discussion section, scorings relate to the impact and appropriateness of the work: How quantitatively (numerically) the proposal overcomes the other ones given in literature? What novelty does the author give to? How high does the proposal improve the system/model to solve the problem? Why did not the other methods in literature achieve this results? The last one is catchy, does it? 
    • Otherwise, when all of above corrections appears on document, author should re-think on the proposal from zero. However, it was a good 2-year exercise, don't you think? 
  7. Conclusions. Finally, the achievement of the proposal, the impact and further works gather into the conclusion section. As authors may know, a full relationship between introduction and conclusion should be achieved: every goal (either main or specific) in the introduction section must be accomplished on the conclusion section. Normally, these two bonded sections are the shortest ones of the document. In the undergraduate classroom experience, it corresponds to the showing-out of the exam scores. Well done, you students.