In this study we perform an analysis of the volatility of the budget deficit for EU countries. We address this issue starting from the new requirements of fiscal discipline imposed by the Treaty on Stability, Coordination and Governance adopted by 25 European Union member states and taking into account the economic crisis impact. The major purpose of this study is to identify the most significant determinants of budget deficit volatility in a comparative study for old EU member states and New Member States (NMS). This study aims to test the impact of macroeconomic variables such as public expenditures, economic growth rate, and unemployment on the budget balance volatility, based on panel data. The final purpose of the article is to reveal the strategies to stop the immense increase in fiscal deficits and to regain fiscal stability to fulfil the new rules of fiscal governance. We anticipate that the implementation of this new fiscal discipline requires a more efficient public sector for both old and NMS and a reconsideration of state intervention in the economy.