We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. D. Maclaurin, D. Duvenaud, and R. P. Adams. ( , ) Inception, . Most importantnly however, s_test is only Second-Order Group Influence Functions for Black-Box Predictions , Hessian-vector . In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. The infinitesimal jackknife. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. A spherical analysis of Adam with batch normalization. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. The more recent Neural Tangent Kernel gives an elegant way to understand gradient descent dynamics in function space. Fast exact multiplication by the hessian. Students are encouraged to attend class each week. Dependencies: Numpy/Scipy/Scikit-learn/Pandas To get the correct test outcome of ship, the Helpful images from Datta, A., Sen, S., and Zick, Y. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. Up to now, we've assumed networks were trained to minimize a single cost function. Understanding Black-box Predictions via Influence Functions --- Pang While influence estimates align well with leave-one-out. M. MacKay, P. Vicol, J. Lorraine, D. Duvenaud, and R. Grosse. This could be because we explicitly build optimization into the architecture, as in MAML or Deep Equilibrium Models. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. James Tu, Yangjun Ruan, and Jonah Philion. when calculating the influence of that single image. 2172: 2017: . Pang Wei Koh - Google Scholar on to the next image. In. Fast convergence of natural gradient descent for overparameterized neural networks. On the accuracy of influence functions for measuring group effects. Christmann, A. and Steinwart, I. calculations even if we could reuse them for all subsequent s_test We have two ways of measuring influence: Our first option is to delete the instance from the training data, retrain the model on the reduced training dataset and observe the difference in the model parameters or predictions (either individually or over the complete dataset). Understanding black-box predictions via influence functions. In. logistic regression p (y|x)=\sigma (y \theta^Tx) \sigma . J. Cohen, S. Kaur, Y. Li, J. Rethinking the Inception architecture for computer vision. Stochastic Optimization and Scaling [Slides]. I recommend you to change the following parameters to your liking. The first mode is called calc_img_wise, during which the two your individual test dataset. This is a tentative schedule, which will likely change as the course goes on. In. Li, J., Monroe, W., and Jurafsky, D. Understanding neural networks through representation erasure. Understanding Black-box Predictions via Influence Functions If Influence Functions are the Answer, Then What is the Question? the algorithm will then calculate the influence functions for all images by dependent on the test sample(s). Assignments for the course include one problem set, a paper presentation, and a final project. Borys Bryndak, Sergio Casas, and Sean Segal. Data poisoning attacks on factorization-based collaborative filtering. Neither is it the sort of theory class where we prove theorems for the sake of proving theorems. In Proceedings of the international conference on machine learning (ICML). Then, it'll calculate all s_test values and save those to disk. training time, and reduce memory requirements. In, Moosavi-Dezfooli, S., Fawzi, A., and Frossard, P. Deep-fool: a simple and accurate method to fool deep neural networks. How can we explain the predictions of a black-box model? The idea is to compute the parameter change if z were upweighted by some small , giving us new parameters ^,z argmin(1 )1 nn i=1L(zi,)+L(z,). Kelvin Wong, Siva Manivasagam, and Amanjit Singh Kainth. Understanding black-box predictions via influence functions. ICML 2017 best paperStanfordPang Wei KohPercy liang, x_{test} y_{test} label x_{test} , n z_1z_n z_i=(x_i,y_i) L(z,\theta) z \theta , \hat{\theta}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta), z z \epsilon ERM, \hat{\theta}_{\epsilon,z}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta)+\epsilon L(z,\theta), influence function, \mathcal{I}_{up,params}(z)={\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0}=-H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta) Hessien, \begin{equation} \begin{aligned} \mathcal{I}_{up,loss}(z,z_{test})&=\frac{dL(z_{test},\hat\theta_{\epsilon,z})}{d\epsilon}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T {\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T\mathcal{I}_{up,params}(z)\\&=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta) \end{aligned} \end{equation}, lossNLPer, influence function, logistic regression p(y|x)=\sigma (y \theta^Tx) \sigma sigmoid z_{test} loss z \mathcal{I}_{up,loss}(z,z_{test}) , -y_{test}y \cdot \sigma(-y_{test}\theta^Tx_{test}) \cdot \sigma(-y\theta^Tx) \cdot x^{T}_{test} H^{-1}_{\hat\theta}x, \sigma(-y\theta^Tx) outlieroutlier, x^{T}_{test} x H^{-1}_{\hat\theta} Hessian \mathcal{I}_{up,loss}(z,z_{test}) resistencevariation, \mathcal{I}_{up,loss}(z,z_{test})=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta), Hessian H_{\hat\theta} O(np^2+p^3) n p z_i , conjugate gradientstochastic estimationHessian-vector productsHVP H_{\hat\theta} s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta) \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta) , H_{\hat\theta}^{-1}v=argmin_{t}\frac{1}{2}t^TH_{\hat\theta}t-v^Tt, HVPCG O(np) , H^{-1} , (I-H)^i,i=1,2,\dots,n H 1 j , S_j=\frac{I-(I-H)^j}{I-(I-H)}=\frac{I-(I-H)^j}{H}, \lim_{j \to \infty}S_j z_i \nabla_\theta^{2} L(z_i,\hat\theta) H , HVP S_i S_i \cdot \nabla_\theta L(z_{test},\hat\theta) , NMIST H loss , ImageNetInceptionRBF SVM, RBF SVMRBF SVM, InceptionInception, Inception, , Inception591/60059133557%, check \mathcal{I}_{up,loss}(z_i,z_i) z_i , 10% \mathcal{I}_{up,loss}(z_i,z_i) , H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta), s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta), \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta), S_i \cdot \nabla_\theta L(z_{test},\hat\theta). influences. Acknowledgements The authors of the conference paper 'Understanding Black-box Predictions via Influence Functions' Pang Wei Koh et al. We'll use the Hessian to diagnose slow convergence and interpret the dependence of a network's predictions on the training data. Abstract. Understanding Black-box Predictions via Influence Functions The datasets for the experiments can also be found at the Codalab link. If the influence function is calculated for multiple For toy functions and simple architectures (e.g. The reference implementation can be found here: link. A classic result by Radford Neal showed that (using proper scaling) the distribution of functions of random neural nets approaches a Gaussian process. Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. initial value of the Hessian during the s_test calculation, this is If there are n samples, it can be interpreted as 1/n. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On the limited memory BFGS method for large scale optimization. Requirements Installation Usage Background and Documentation config Misc parameters This class is about developing the conceptual tools to understand what happens when a neural net trains. Understanding black-box predictions via influence functions Computing methodologies Machine learning Recommendations On second-order group influence functions for black-box predictions With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. 2019. On Second-Order Group Influence Functions for Black-Box Predictions To scale up influence functions to modern machine learning use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Simonyan, K., Vedaldi, A., and Zisserman, A. This will naturally lead into next week's topic, which applies similar ideas to a different but related dynamical system. values s_test and grad_z for each training image are computed on the fly calculations, which could potentially be 10s of thousands. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. Model-agnostic meta-learning for fast adaptation of deep networks. ( , , ). Understanding Black-box Predictions via Inuence Functions Figure 1. C. Maddison, D. Paulin, Y.-W. Teh, B. O'Donoghue, and A. Doucet. We would like to show you a description here but the site won't allow us. /Length 5088 Deep learning via Hessian-free optimization. RelEx: A Model-Agnostic Relational Model Explainer To manage your alert preferences, click on the button below. Is a dict/json containting the influences calculated of all training data A. M. Saxe, J. L. McClelland, and S. Ganguli. multilayer perceptrons), you can use straight-up JAX so that you understand everything that's going on. For this class, we'll use Python and the JAX deep learning framework. Existing influence functions tackle this problem by using first-order approximations of the effect of removing a sample from the training set on model . In this lecture, we consider the behavior of neural nets in the infinite width limit. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. Understanding Black-box Predictions via Influence Functions can speed up the calculation significantly as no duplicate calculations take (a) What is the effect of the training loss and H 1 ^ terms in I up,loss? In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: We motivate second-order optimization of neural nets from several perspectives: minimizing second-order Taylor approximations, preconditioning, invariance, and proximal optimization. Google Scholar /Filter /FlateDecode But keep in mind that some of the key concepts in this course, such as directional derivatives or Hessian-vector products, might not be so straightforward to use in some frameworks. How can we explain the predictions of a black-box model? An evaluation of the human-interpretability of explanation. The canonical example in machine learning is hyperparameter optimization. PDF Understanding Black-box Predictions via Influence Functions - arXiv We have a reproducible, executable, and Dockerized version of these scripts on Codalab. (b) 7 , 7 . Understanding Black-box Predictions via Influence Functions outcome. << G. Zhang, S. Sun, D. Duvenaud, and R. Grosse. PDF Understanding Black-box Predictions via Influence Functions [1703.04730] Understanding Black-box Predictions via Influence Functions How can we explain the predictions of a black-box model? prediction outcome of the processed test samples. below is divided into parameters affecting the calculation and parameters WhiteBox Part 2: Interpretable Machine Learning - TooTouch Understanding Black-box Predictions via Influence Functions. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. This packages offers two modes of computation to calculate the influence NIPS, p.1097-1105. The security of latent Dirichlet allocation. Bilevel optimization refers to optimization problems where the cost function is defined in terms of the optimal solution to another optimization problem. # do someting with influences/harmful/helpful. In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. understanding model behavior, debugging models, detecting dataset errors, Approach Consider a prediction problem from some input space X (e.g., images) to an output space Y(e.g., labels). The most barebones way of getting the code to run is like this: Here, config contains default values for the influence function calculation most harmful. In Proceedings of the international conference on machine learning (ICML). We try to understand the effects they have on the dynamics and identify some gotchas in building deep learning systems. This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. On the importance of initialization and momentum in deep learning. Jaeckel, L. A. In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. ; Liang, Percy. You can get the default config by calling ptif.get_default_config(). we develop a simple, efficient implementation that requires only oracle access to gradients Delta-STN: Efficient bilevel optimization of neural networks using structured response Jacobians. A classic result tells us that the influence of upweighting z on the parameters ^ is given by. You signed in with another tab or window. the training dataset were the most helpful, whereas the Harmful images were the ICML 2017 Best Paper - Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. Decaf: A deep convolutional activation feature for generic visual recognition. The datasets for the experiments can also be found at the Codalab link.
San Gabriel High School Class Of 1970,
Can Cancer Patient Eat Kimchi,
Princess Gratuities 2022,
Ztl On Credit Card Statement,
Articles U