For the purpose of improving the quality of services, softwares or online services are collecting various of user data, such as personal information and locations. Such data facilitates mining statistical knowledge of users, but threatens users’ privacy as it may reveal sensitive information (e.g., identities and activities) about individuals. This work considers distribution estimation over user-contributed data meanwhile providing rigid protection of their data with local ε-differential privacy (ε-LDP), which sanitizes each user's data on the client's side (e.g, on the user's mobile device). Our privacy protection covers both qualitative data (e.g., categorical data) and discrete quantitative data (e.g., location data). Specifically, for categorical data, we derive an optimal ε-LDP mechanism (termed as k-subset mechanism) from mutual information perspective, and further show its optimality over existing approaches within the context of discrete distribution estimation; for discrete quantitative data that have arbitrary distance metric, we provide an efficient extension of k-subset mechanism by proposing a variant of the popular Exponential Mechanism (EM) to tackle the asymmetry issue on the data domain. Experiments on real-world datasets and simulated scenarios show that our mechanism is highly efficient and reduces nearly a fraction of exp(-ε2) error for distribution estimation when compared to existing approaches.