A multi-objective genetic algorithm approach is formulated to optimize the design of silicon-photonics complex circuits with contradicting performance metrics and no closed-form expression for the circuit performance. A case study is the interleaver/deinterleaver circuit which mixes/separates optical signals into/from different physical channels while preserving the wavelength-division-multiplexing specifications. These specifications are given as channel spacing of 50 GHz, channel 3-dB bandwidth of at least 20 GHz, channel free spectral range of 100 GHz, crosstalk of −23 dB or less, and signal dispersion less than 30 ps/nm. The essence of the proposed approach lies in the formulation of the fitness functions and the selection criteria to optimize the values of the three coupling coefficients, which govern the circuit performance, in order to accommodate the contradicting performance metrics of the circuit. The proposed approach achieves the optimal design in an incomparably short period of time when contrasted with the previous tedious design method based on employing Z-transform and visual inspection of the transmission poles and zeros.