We investigate the performance of receiver diversity techniques by considering energy detector based optimal spectrum sensing in cognitive radio networks over Rayleigh fading channel for both non-cooperative and cooperative communication systems. The analysis is first focused on the non-cooperative spectrum sensing part, including the analyses of selection combining (SC), equal gain combining (EGC), and maximal ratio combining (MRC) receiver diversity systems. We derive exact closed-form expressions for the optimal detection thresholds for each diversity scheme. By using these expressions, we present their detection performance compared to the no-diversity case. Then in the second part of this paper, the performance analysis of the proposed system is investigated for amplify-and-forward relaying spectrum sensing with SC, EGC, and MRC diversity techniques in Rayleigh fading channels. Particularly, average detection probability of end-to-end signal-to-noise ratio for SC and EGC schemes and the exact closed-form expressions of the optimal detection threshold for SC, EGC, and MRC schemes are derived. We also present detection performance for the cooperative system compared to the non-cooperative one. The analyses are validated by simulated receiver operating characteristic curves.