This paper proposes a system design for automation in multi-agent-based manufacturing systems to conduct a given complex task automatically by controlling multiple robotic manipulators in a systematic manner. To this end, the proposed system is designed with three-module configurations: environmental perception, task planning, and motion planning. The environmental perception module utilizes a vision sensor to recognize all objects placed on the workspace and extract their unique ID, size, and pose. The task planning module divides a given task into primitive skill levels and distributes each primitive skill to the associated robotic manipulator with the relevant object information in a systematic manner for robotic manipulators not to collide with each other. The motion planning module determines the motion of a robotic arm and a robotic hand by solving inverse kinematics for the robotic arm and by opening or closing two fingers. The proposed system has been tested and verified in real robot environments through a complex task "peg in hole" that requires at least two robotic manipulators.