Each decision-making, a fundamental human process, has been more and more supported by computer systems in the second part of the last century the so-called decision support systems (DSS). In the twenty-first century, intelligent decision support systems (IDSS) utilize artificial intelligence (AI) techniques to enhance and improve support for decision-makers. Ethics, as the branch of philosophy dealing with concepts of right and wrong conduct, was solely attributed to human intelligence, behaviour or decision. With artificial intelligence-based systems, the notion AI Ethics (or machine ethics) refers to ethical decision-making by machines. Specifically, in the field of IDSS, where AI techniques participate (at least partially) in the decision-making process, it is of utmost importance to design such systems with ethical considerations. In this paper, we present a classification of ethical values as an underlying concept for a framework consisting in a set of rules that need to be part of any IDSS design. Applied to a decision process for candidate pre-selection (e.g. hiring), we illustrate the pitfalls that lead to biased decisions and methods for mitigation of “wrong decisions”, from an ethical point of view. We suggest how to operationalize Ethics by Design and how to approach Ethics Build-In AI-based systems.