We show that the Wald-Zoupas prescription for gravitational charges is valid in the presence of anomalies and field-dependent diffeomorphism, but only if these are related to one another in a specific way. The geometric interpretation of the allowed anomalies is exposed looking at the example of BMS symmetries; they correspond to soft terms in the charges. We determine if the Wald-Zoupas prescription coincides with an improved Noether charge. The necessary condition is a certain differential equation, and when it is satisfied, the boundary Lagrangian of the resulting improved Noether charge contains, in general, a nontrivial corner term that can be identified a priori from a condition of anomaly freeness. Our results explain why the Wald-Zoupas prescription works in spite of the anomalous behavior of BMS transformations, and should be helpful to relate different branches of the literature on surface charges.