. As always, I really love the work that you are doing. I wanted to ask you a question. I have a high enough rate limit for the Perspective API to do real-time scoring on the data as it comes in. I am writing the code now for scoring Reddit, but I'm unsure on how..
Conversation
.. to pre-process the Reddit comments before scoring the text. I was thinking two main things would be to remove links from the text and also to make sure that any quoted text is also removed (if a child comment author is quoting the parent comment.
Any suggestions on this?
Replying to
Thanks - likewise, I love your work too! This sounds like a great idea... I'm sure that scores from the Perspective API would be helpful for researchers looking at hate speech, online conversations, etc.
Replying to
I don't know much about the Perspective API but I would suggest looking at the input features of the model (if specs are available), and making sure that the Reddit comment text contains what is useful for their model. E.g., if they actually parse URLs, then maybe URLS are good?
1
Replying to
Also, removing quoted text is a neat idea, but I wonder if such text contains semantic information relevant to the Perspective API? E.g. if someone uses racial hate speech, then another user quotes it (and agrees with it), then they are diffusing the hate speech...
2
1
Replying to
Great response! I really appreciate your input. I think I will reach out to their dev team and ask them how best to handle these type of situations. As you said, Perspective may account for URLS and do something special for them.
2
1
Show replies