The recent development of advanced black box optimization algorithms has promised order of magnitude improvements in optimization speed when solving accelerator physics problems. However in practice, these algorithms remain inaccessible to the general accelerator community, due to the expertise and infrastructure required to apply them towards solving optimization problems. In this work, we introduce the Python package, Xopt, which implements a simple interface for connecting arbitrarily specified optimization problems with advanced optimization algorithms. Users specify optimization problems and algorithms with a minimal Python script, allowing flexible interfacing with both experimental online control systems and simulated design problems, while minimizing the need for algorithmic expertise or software development. We describe case-studies where cutting-edge Bayesian optimization and genetic algorithms implemented in Xopt are used to solve online control and accelerator design problems. The same algorithms are also used to solve simulated optimization problems in high performance computing clusters using the same interface.