Reviews should be published as supplementary documents to a paper. They should be considered as a minor publication under the reviewer’s name.
We should hold reviewers accountable. If we allow authors to rate, reviewers who give high scores will be rated higher, rather than ones that are objective. May be limit rating to only AC and other reviewers? As AC, I have tried to call out unethical reviewers, but not easy.https://twitter.com/DocXavi/status/1191793574909763584 …
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
May be I am naive but why even rely on peer reviewers? Just start a blog and put your work out there. I am not an academic but know a few that are. Seems like journals exists for the sole purpose of helping someone check a box to get tenure.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Yep, limiting to AC may be the safest option, extending it to other reviewers does not seem too dangerous to me either. Also allowing reviewers to publish or not their scores should be possible, otherwise it would be even harder to find reviewers.
-
in that case though we should avoid COIs between an AC and her/his reviewers at any levels, loopholes in a quality checking system are catastrophic
End of conversation
New conversation -
-
-
The problem is volume. With that many papers submitted, it becomes a stochastic process. One solution is for authors submit less but better papers. I know this is not gonna happen. I like to dream tough.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Adding more ratings is not the solution. We have way too many ratings everywhere these days. I personally find double-blind more fair than single-blind. Some journals experiment with all names open, but my fear is that will lead to junior never criticizing senior because afraid.
-
What about pseudonymous reviews? The pseudonym can gain reputation - perhaps even across years and conferences. Pseudonymity is widely used among Wikipedia editors.
End of conversation
New conversation -
-
-
The underlying problem is strong limitation of accepted papers (are only 20% of submissions good enough?). No matter if you give ratings between 1 and 10 or have an Uber system with 5 or no 5, it is going to be unfair and random in many cases because of competition.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
#AI is big and important enough and still has enough growth potential in industry and society that we can let each other live. More diversification in topics, not everyone shooting for the same conference, and some journals that make sense could help.Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
A different approach would be for the reviewers to rate each other. I usually read other reviewers' review once the decision is out to see if I have missed anything and/or if they agree with me. There is no (or minimal) conflict of interest as well.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.