I need assistance from people involved in academic research (especially those who use my datasets for research purposes). Before I release the next version of the Pushshift API, I need to create procedures and a privacy statement that deals with users who request to be removed
Conversation
from the data dumps and API. I need help in creating cogent disclosure statements that let researchers know that data was removed at the request of a user while retaining some metadata within the datasets. If anyone can assist me with this, it would be greatly appreciated.
1
2
My thoughts on this are censoring out the author name first and foremost -- but I don't know if the body content itself should also be removed. I would also need to create addendums along with the monthly data dumps detailing how data was modified to keep researchers informed.
Replying to
Several Qs? What is the motivation for this? The data being collected is/was inherently published in public to begin with right? How do you validate a request actual comes from the author?
This Tweet was deleted by the Tweet author. Learn more
Replying to
Thanks for the suggestion! I will definitely take a look at how that is handled on that level. I'm trying to balance respecting requests for privacy with academic studies that use the data --I am finding that it's a delicate thing to handle and do correctly.
Replying to
Hi Jason, if you ever want to come up to Princeton and talk through these questions with the folks at the Center for IT Policy, just let me know. We would be happy to help.
1
1
Replying to
I would like that very much! I'll eventually schedule time to come up there!
1