Cooperative transportation is one of the essential tasks for multi-robot systems to imitate the decentralized systems of social insects. However, in a situation involving an obstacle on the pathway, multiple robots need to realize transportation and obstacle removal simultaneously. To address this multitasking problem, we first introduce a learning scenario and train robots’ decentralized policies via multi-agent reinforcement learning. Next, we propose two virtual experiments with blindfold teams and homogeneous teams to analyze the individual behaviors of the trained robots. The results showed that three robots with different policies performed two tasks simultaneously as a team. One robot’s policy tended to perform obstacle removal, and the other robots’ policies tended to perform cooperative transportation. Further, the first robot’s policy had the potential to perform two tasks simultaneously depending on the situation. Finally, we demonstrated the trained policies with three ground robots to show the feasibility of the system.