Calorie restriction (CR) is considered one of the most robust ways to extend life span and reduce the risk of age-related diseases, including cancer, as shown in many different organisms, whereas opposite effects have been associated with high fat diets (HFDs). Despite the proven contribution of sirtuins in mediating the effects of CR in longevity, the involvement of these nutrient sensors, specifically, in the diet-induced effects on tumorigenesis has yet to be elucidated. Previous studies focusing on SIRT1, do not support a critical role for this sirtuin family member in CR-mediated cancer prevention. However, the contribution of other family members which exhibit strong deacetylase activity is unexplored. To fill this gap, we aimed at investigating the role of SIRT2 and SIRT3 in mediating the anti and pro-tumorigenic effect of CR and HFD, respectively. Our results provide strong evidence supporting distinct, context-dependent roles played by these two family members. SIRT2 is indispensable for the protective effect of CR against tumorigenesis. On the contrary, SIRT3 exhibited oncogenic properties in the context of HFD-induced tumorigenesis, suggesting that SIRT3 inhibition may mitigate the cancer-promoting effects of HFD. Given the different functions regulated by SIRT2 and SIRT3, unraveling downstream targets/pathways involved may provide opportunities to develop new strategies for cancer prevention.