Sorry that the previous Y axis scale shifted between the comparisons. Here they are side by side with the same scale.
Conversation
Replying to
Note that, at least in previous version of API, simple identity markers scored higher on toxicity, making for a biased distribution of false positives. Thus, it's possible that simple statement of Jewish identity or pro-Jewish posts will be misclassified engadget.com/2017/09/01/goo
2
2
Replying to
That's a good point. I know the model is sort of a "magic black box" to all of us right now -- but one thing that will help determine and address these biases is to do a parallel comparison with other social media platforms. I am going to score Reddit for the month of November ..
And see what the baseline toxicity score is for Reddit. I will also run the "jew,jews,jewish" query to see the results for Reddit. If the model is flawed in a fundamental way, I would expect to see a likewise double of the toxicity percentage for Reddit ... But if Reddit gets ..
1
1
.. a baseline toxicity score of say 20% and then the jew toxicity avg is 24%, then that gives us more information and that the bias (if it exists) for those Jewish terms isn't so much such that it makes the Gab comparison unusable. There is a lot of data to score, but this will..
1
Show replies
Replying to
Good test might be to compare posts using "Jewish" in subreddits celebrating Jewish culture with those on politics, etc. Could give leverage on the question how much if the toxicity score is due to the word "Jewish" vs it's context.
1
2
Replying to
👍Absolutely -- I'm really excited to get Reddit scored because there are a lot of tests we can run against Reddit data and as you said, subreddits will help us "compartmentalize" researching various communities, etc.
2