This paper presents a learning classifier system ensemble for knowledge discovery from incremental data. The new ensemble was designed with a two-level architecture to improve the generalization ability. The new incoming cases are first bootstrapped to generate data as inputs to the first level classical learning classifier systems. The second level contains a plurality-vote module to determine the final classification by combining the classification results of the first level learning classifier systems. Each learning classifier system in the first level consists of two major modules, a genetic algorithm module for facilitating rule-discovery and a reinforcement learning module for adjusting the strength of the corresponding rules when rewards are received from the environment. We propose a revised Wilson's compact rule algorithm for generation of the compact rule set from the population set to improve the readability of the model. Two experiments were conducted. One was data mining of medical data and the other was steganalysis of images. The experimental results have shown that the new ensemble produced better performance on incremental data mining and better generalization than the single learning classifier system and other supervised learning methods. The results also showed that the compact rules were more interpretable.