ChatGPT as a Reviewer

A very useful tips of ChatGPT 4: We can use it as an external reviewers for our own papers. This is a very recent finding of mine along my way of writing reviews. I used ChatGPT as an assistant (please do not have it write it all for you, which is clearly forbidden). I noticed that despite the fact that it clear does not posses a humanoid taste of HCI or general CS research and high-level thinking that critically assesses a piece of research (which is why we are needed), it suffices to provide logical critique and importantly it can critically evaluate the evaluation sections of papers. It can reveal whether our papers are well-written enough to make people understand the logic and flow of the thinking behind the characters. It can also tell whether there are flaws in the statistical tests we used and what kind of tests we should have done if flaws exist. It tells us what is missing to prove every point we’ve made in the paper. Since it is prompt-based communication, we can actually adjust the difficulty of ChatGPT as our reviewers of the paper by telling it to be more critical (mostly likely, get prepared for the worst) or more generous (less preferred tho). The best about ChatGPT as an external reviewer is that it is always a third opinion. Once you open up a new dialogue it loses its memory about your previous discussion and it gets to reassess your paper as if it’s never read it before. We know what is critical of our manuscripts, if the same concerns keep showing up in this iterations.

IMO, peer-review system is always a game of luck. We will still need to roll the dice even if the paper is complete. But what does it take to make the paper complete? I believe that the dice would not be fair until we make our manuscript to a level where the logic is seamless, the stats are solid, and the presentations are connective and good, or as good as we can be. Then the reviewers can only criticize the ideas. The whole point is to shrink the space where they can attack our submissions. We cannot control whether they like the idea or not but the completeness of the paper. I suggest everyone try using ChatGPT to write a review of our own paper once and we will see how helpful it is.

BTW, if you are writing reviews for others using ChatGPT, please make sure you read and write the very first version without the help of ChatGPT or do not use ChatGPT at all (please strictly follow the protocal of writing a review). The closure of thinking is what matters here afterall. Otherwise, it will ruin the whole point of a peer-review system.