Time-evolution of the Universe as described by the Friedmann equation can be coupled to equations of motion of matter fields. Quantum effects may be incorporated to improve these classical equations of motion by the renormalization group (RG) running of their couplings. Since temporal and thermal evolutions are linked to each other, astrophysical and cosmological treatements based on zero-temperature RG methods require the extension to finite-temperatures. We propose and explore a modification of the usual finite-temperature RG approach by relating the temperature parameter to the running RG scale as (in natural units), where is acting as a running cutoff for thermal fluctuations and the momentum can be used for the quantum fluctuations. In this approach, the temperature of the expanding Universe is related to the dimensionless quantity (and not to ). We show that by this choice dimensionless RG flow equations have no explicit -dependence, as it is convenient. We also discuss how this modified thermal RG is used to handle high-energy divergences of the RG running of the cosmological constant and to "solve the triviality" of the model by a thermal phase transition in terms of in Euclidean dimensions.