Despite their apparent simplicity, suspensions of hard spheres in a Newtonian fluid show complex non-Newtonian behaviors and remain poorly understood. Recent works have pointed out the crucial role of interparticle contact forces in these behaviors. Here, we show that the same (polystyrene) particles, when immersed in different Newtonian solvents, show different behaviors at both the microscopic and macroscopic scales. Thanks to interparticle force measurements in each solvent together with rheological measurements, we show how the fine details of the pairwise particle interactions impact the macroscopic behavior. The rheological properties (shear thinning, shear thickening, jamming solid fraction value) of the suspensions, made up of same particles, are shown to depend on the nature of the solvent. Here, we highlight several mechanisms at the particle scale: the swelling of polymeric particles in an organic solvent, the role of colloidal repulsive forces and inertia for particles in a water solution, and the variation of the friction coefficient as a function of the load for particles immersed in silicone oils. Our study provides new quantitative data to test micromechanical models and simulations. It questions the interpretation of previous experimental works. Finally, it shows the need to systematically characterize the interparticle normal and tangential forces when studying a given suspension of hard spheres in a Newtonian fluid