In this paper, room temperature forging (RTF, 25 °C) and warm forging (WF, 200 °C) of Al-Mg-Si alloy were performed, for strengthening the 6082 Al alloy at low applied strain. Overall WF-AlMgSi alloy forged just after 2 cycles (1.26 true strain) exhibits maximum hardness, tensile strength, J-integral value, and plastic strain in all investigated samples. This is due to: (i) the formation of ultrafine grain (UFG) of size 411 nm, (ii) the formation of stress free dynamic recrystallized (DRX) grains, and (iii) the complete precipitation of β"-phase. The UFG increased the hardness and yield strength (YS) according to the conventional Hall-Petch effect, and the β"-phase increased the hardness and the YS by providing the obstacle to the movement of mobile dislocations. The stress free DRX grains accommodate a large number of mobile dislocations, which further enhances the plastic strain. The UFG, the DRX grains, and the β"-phase effect, all combined together further enhance the J-integral value. The effects of RTF, WF and strains on the emergence of various forms of shear bands were also discussed in this paper. Softening effect on both RTF and WF-AlMgSi alloy after processing beyond 2 cycles is due to factors such as saturation of strengthening through Hall-Petch effect and coarsening of β"-phase in the RTF-AlMgSi alloy and β'-phase in the WF-AlMgSi alloy, respectively.