Assessing the performance of algorithms in solving building energy optimization (BEO) problems with different properties is essential for selecting appropriate algorithms to achieve the best design solution. This study begins with a classification of the properties of BEO problems from three perspectives, namely, design variables, objective functions, and constraints. An analytical approach and a numerical approach are proposed to determine the properties of BEO problems. Six BEO test problems with different properties, namely, continuous vs. discrete, convex vs. non-convex, linear vs. non-linear, uni-modal vs. multimodal, and single-dimensional vs. multi-dimensional, are composed to evaluate the performance of algorithms. The selected optimization algorithms for performance assessment include the discrete Armijo gradient, Particle Swarm Optimization (PSO), Hooke-Jeeves, and hybrid PSO and Hooke-Jeeves. The assessment results indicate that multimodality can cause Hooke-Jeeves and discrete Armijo gradient algorithms to fall into local optima traps. The convex, non-convex, linear and non-linear properties of uni-modal BEO problems have little impact on the performance behavior of the algorithms. The discrete Armijo gradient and Hooke-Jeeves are not recommended for solving discrete and multi-dimensional BEO problems.