We generalize factor analysis models by allowing the concentration matrix of the residuals to have nonzero off-diagonal elements. The resulting model is named graphical factor analysis model. Allowing a structure of associations gives information about the correlation left unexplained by the unobserved variables, which can be used both in the confirmatory and exploratory context. We first present a sufficient condition for global identifiability of this class of models with a generic number of factors, thereby extending the results in Stanghellini (1997) and Vicard (2000). We then consider the issue of model comparison and show that fast local computations are possible for this purpose, if the conditional independence graphs on the residuals are restricted to be decomposable and a Bayesian approach is adopted. To achieve this aim, we propose a new reversible jump MCMC method to approximate the posterior probabilities of the considered models. We then study the evolution of political democracy in 75 developing countries based on eight measures of democracy in two different years.