Probability of informed trading (PIN) models characterize trading with certain types of information through a tree structure. Different tree structures with different numbers of groups for market participants have been proposed, with no clear, consistent tree used in the literature. One of the main causes of this inconsistency is that these trees are artificially proposed through a bottom-up approach rather than implied by actual market data. Therefore, in this paper, we propose a method that infers a tree structure directly from empirical data. More precisely, we use hierarchical clustering to construct a tree for each individual firm and then infer an aggregate tree through a voting mechanism. We test this method on US data from January 2002 for 7608 companies, which results in a tree with two layers and four groups. The characteristics of the resulting aggregate tree are between those of several proposed tree structures in the literature, demonstrating that these proposed trees all reflect only part of the market, and one should consider the proposed empirically driven method when seeking a tree representing the whole market.