The right to be forgotten states that a data owner has the right to erase
their data from an entity storing it. In the context of machine learning (ML),
the right to be forgotten requires an ML model owner to remove the data owner’s
data from the training set used to build the ML model, a process known as
machine unlearning. While originally designed to protect the privacy of the
data owner, we argue that machine unlearning may leave some imprint of the data
in the ML model and thus create unintended privacy risks. In this paper, we
perform the first study on investigating the unintended information leakage
caused by machine unlearning. We propose a novel membership inference attack
that leverages the different outputs of an ML model’s two versions to infer
whether a target sample is part of the training set of the original model but
out of the training set of the corresponding unlearned model. Our experiments
demonstrate that the proposed membership inference attack achieves strong
performance. More importantly, we show that our attack in multiple cases
outperforms the classical membership inference attack on the original ML model,
which indicates that machine unlearning can have counterproductive effects on
privacy. We notice that the privacy degradation is especially significant for
well-generalized ML models where classical membership inference does not
perform well. We further investigate four mechanisms to mitigate the newly
discovered privacy risks and show that releasing the predicted label only,
temperature scaling, and differential privacy are effective. We believe that
our results can help improve privacy protection in practical implementations of
machine unlearning. Our code is available at
https://github.com/MinChen00/UnlearningLeaks.

Go to Source of this post
Author Of this post: <a href="http://arxiv.org/find/cs/1/au:+Chen_M/0/1/0/all/0/1">Min Chen</a>, <a href="http://arxiv.org/find/cs/1/au:+Zhang_Z/0/1/0/all/0/1">Zhikun Zhang</a>, <a href="http://arxiv.org/find/cs/1/au:+Wang_T/0/1/0/all/0/1">Tianhao Wang</a>, <a href="http://arxiv.org/find/cs/1/au:+Backes_M/0/1/0/all/0/1">Michael Backes</a>, <a href="http://arxiv.org/find/cs/1/au:+Humbert_M/0/1/0/all/0/1">Mathias Humbert</a>, <a href="http://arxiv.org/find/cs/1/au:+Zhang_Y/0/1/0/all/0/1">Yang Zhang</a>

By admin