Iterative assessment and improvement of dnn operational accuracy

A Guerriero, R Pietrantuono… - 2023 IEEE/ACM 45th …, 2023 - ieeexplore.ieee.org
2023 IEEE/ACM 45th International Conference on Software …, 2023ieeexplore.ieee.org
Deep Neural Networks (DNN) are nowadays largely adopted in many application domains
thanks to their human-like, or even superhuman, performance in specific tasks. However,
due to unpredictable/unconsidered operating conditions, unexpected failures show up on
field, making the performance of a DNN in operation very different from the one estimated
prior to release. In the life cycle of DNN systems, the assessment of accuracy is typically
addressed in two ways: offline, via sampling of operational inputs, or online, via pseudo …
Deep Neural Networks (DNN) are nowadays largely adopted in many application domains thanks to their human-like, or even superhuman, performance in specific tasks. However, due to unpredictable/unconsidered operating conditions, unexpected failures show up on field, making the performance of a DNN in operation very different from the one estimated prior to release.In the life cycle of DNN systems, the assessment of accuracy is typically addressed in two ways: offline, via sampling of operational inputs, or online, via pseudo-oracles. The former is considered more expensive due to the need for manual labeling of the sampled inputs. The latter is automatic but less accurate.We believe that emerging iterative industrial-strength life cycle models for Machine Learning systems, like MLOps, offer the possibility to leverage inputs observed in operation not only to provide faithful estimates of a DNN accuracy, but also to improve it through remodeling/retraining actions.We propose DAIC (DNN Assessment and Improvement Cycle), an approach which combines "low-cost" online pseudo-oracles and "high-cost" offline sampling techniques to estimate and improve the operational accuracy of a DNN in the iterations of its life cycle. Preliminary results show the benefits of combining the two approaches and integrating them in the DNN life cycle.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果