Woodfisher: Efficient second-order approximation for neural network compression
SP Singh, D Alistarh - Advances in Neural Information …, 2020 - proceedings.neurips.cc
Advances in Neural Information Processing Systems, 2020•proceedings.neurips.cc
Second-order information, in the form of Hessian-or Inverse-Hessian-vector products, is a
fundamental tool for solving optimization problems. Recently, there has been significant
interest in utilizing this information in the context of deep neural networks; however,
relatively little is known about the quality of existing approximations in this context. Our work
considers this question, examines the accuracy of existing approaches, and proposes a
method called WoodFisher to compute a faithful and efficient estimate of the inverse …
fundamental tool for solving optimization problems. Recently, there has been significant
interest in utilizing this information in the context of deep neural networks; however,
relatively little is known about the quality of existing approximations in this context. Our work
considers this question, examines the accuracy of existing approaches, and proposes a
method called WoodFisher to compute a faithful and efficient estimate of the inverse …
Abstract
Second-order information, in the form of Hessian-or Inverse-Hessian-vector products, is a fundamental tool for solving optimization problems. Recently, there has been significant interest in utilizing this information in the context of deep neural networks; however, relatively little is known about the quality of existing approximations in this context. Our work considers this question, examines the accuracy of existing approaches, and proposes a method called WoodFisher to compute a faithful and efficient estimate of the inverse Hessian.
proceedings.neurips.cc
以上显示的是最相近的搜索结果。 查看全部搜索结果