A comprehensive experimental study on the ejector and cooling performances of a transcritical CO₂ ejector-expansion system is presented. The ejector performance of the entrainment ratio, pressure lift ratio, and efficiency were investigated under various primary flow pressure, secondary flow pressure, and back pressure conditions. A new coefficient of mass balance, β, was introduced based on the liquid mass balance in and/or out of the vapor–liquid separator, which was able to estimate the deviation between current operating state and a steady state. The experiment results show that the coefficient of performance (COP) ratio of the ejector-expansion refrigeration system to the basic system decreases from 18.9% to −11% with the coefficient of mass balance increasing from −0.1 to 0.1. Both the entrainment ratio and COP decrease with an increase in the coefficient of mass balance.