Nowadays, many people take short breaks with their smartphone at work. The decision whether to continue working or to take a smartphone break is a so-called labour versus leisure decision. Motivational models predict that people are more likely to switch from labour (work) to leisure (smartphone) the more fatigue or boredom they experience. In turn, fatigue and boredom are expected to decrease after the smartphone was used. However, it is not yet clear how smartphone use at work relates to fatigue and boredom. In this study, we tested these relationships in both directions. Participants (n = 83, all PhD candidates) reported their current level of fatigue and boredom every hour at work while an application continuously logged their smartphone use. Results indicate that participants were more likely to interact with their smartphone the more fatigued or bored they were, but that they did not use it for longer when more fatigued or bored. Surprisingly, participants reported increased fatigue and boredom after having used the smartphone (more). While future research is necessary, our results (i) provide real-life evidence for the notion that fatigue and boredom are temporally associated with task disengagement, and (ii) suggest that taking a short break with the smartphone may have phenomenological costs.