Concerning security, privacy, anti-abuse, fairness, ethics Security badness: * the MS Tay chatbot has training data poisoning * YouTube filtering videos which aren't appropriate for children, but people will evade
-
-
Prikaži ovu nit
-
Safety badness: research showing that changing the lighting can cause self-driving cars to steer different directions (e.g. off a cliff. Which is bad.)
Prikaži ovu nit -
Privacy: * model inversion attack, where the attacker can recover the training data from queries to the model [ THIS SHOULD BE BETTER KNOWN IT'S A REAL PROBLEM FOR DATA RETENTION AS WELL ]
Prikaži ovu nit -
Fairness and ethics: * fewer people from a set in a group --> worse performance e.g. bad performance at facial recognition for people of colour, women
Prikaži ovu nit -
How can we design training algorithms which provide these notions of trust?
Prikaži ovu nit -
Failed attempt: tried to train a model to recognize handwriting and it has to be not sensitive to small changes [because otherwise will memorize private data], but that opens up to new attacks, like confusing some written 5 and 3. Tried to defend against a specific attack and
Prikaži ovu nit -
Is achieving trustworthy ML any different from computer security? Lots of tradeoffs. Can you win here? In ML can learn with differential privacy. Differential privacy basically means that an attacker can't tell if any particular person is in/out of a data set. [Not typing math]
Prikaži ovu nit -
How to train a model with differential privacy? We can bound how much the model is sensitive to particular data items by bounding the gradient. Add some noise. Voila!
Prikaži ovu nit -
What's interesting about this is that when you have unusual examples of handwritten numbers. If you want to have a model that learns them, it has to have very loose privacy [ because they're unusual and so if there's differential privacy, the model wouldn't "see" them ]
Prikaži ovu nit -
So if you're using differential privacy, you get robust patterns, but not good recognition at the edges [ Note that this means exactly for fairness what you think it does: rare examples will have worse performance so people in unusual situations will have worse performance ]
Prikaži ovu nit -
What about test time? * admission control may address lack of assurance: sandboxing, authentication, etc. to weed out certain kinds of outliers
Prikaži ovu nit -
* model governance -- things like GDPR mean you have to pay a lot of attention once it's been deployed. When a user asks to have their data deleted, do you have to delete all the models that you made? Instead, shard users and do only partial retraining [ cool! ]
Prikaži ovu nit -
Policies are needed to align ML with societal norms: security, privacy, ethics Technology needs to: propose training algorithms which can satisfy policies, at test time have admission control and model governance Beyond tech, complement with legal frameworks and education
Prikaži ovu nit -
Q: you talk about avoiding an arms race for defenders. Do you think it's avoidable? A: I think so. Privacy shows that there isn't necessarily a conflict between requirements and performance.
Prikaži ovu nit -
Q: What would you take from this technical discussion and convey to policy folks/regulators? A: I'll talk about privacy because I'm more familiar there. There's a gap between tech and policy. Talked about example of taking the policy back to the tech and doing it there.
Prikaži ovu nit -
... but something like differential privacy, it's not clear how to map it back to legislation.
Prikaži ovu nit -
Q: when you're talking about the privacy preserving models, it sounded like a benefit was avoiding overfitting. Does avoiding overfitting other ways benefit privacy? A: Generalization/avoiding overfitting won't necessarily help with privacy because they care about average case.
Prikaži ovu nit -
Q: Can you talk about any algorithms where you can train an algorithm per-person and merge them. What are the limitations? A: Several proposals. Main limit is that one person may not have enough data to make a model in the first place and so have to sync across multiple.
Prikaži ovu nit -
Q: perception is that you pay some price in accuracy to get differential privacy and if you did would that be confusing? A: I don't like that framing because it's incentivizing algorithms to overfit to corner cases because of the way it's measured [ Love this! ]
Prikaži ovu nit -
So then you need to figure out how to better measure performance rather than optimising for benchmarks.
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.