The incorporation of mineral particles into a polymer matrix leads to extra properties of the polymeric material that are linked to the homogeneity of the mixture. A fine homogeneity can be obtained by using a latex route where the two components are first dispersed in water and then dried. Here, we show a novel way to make an elastomeric composite material via a bulk destabilization of a binary colloidal dispersion. The mixture is composed of a natural rubber latex, which is naturally stabilized, and carbon black particles that are dispersed in water with the help of surfactants. We observe either a transition towards a weak gel, which is a reversible state, or a transition towards an irreversible elastic solid. These structural transitions are mainly controlled by the surfactant concentration that tunes the interaction between natural rubber and carbon black particles, the latter bridging polymer droplets together. We also show that the weak gel can become an irreversible elastic solid under a high shear. This shear-induced heteroaggregation phenomenon exhibits the features of a thermally activated process.