This letter addresses the challenge of determining the optimal design of a customizable robot for a given task. We present a motion planner that not only prescribes a path, but also synthesizes the robot design to follow that path. Our approach treats design variables as part of an inverse kinematics problem to jointly optimize manipulator design parameters and trajectory. We apply our method to two distinct problems where rapid prototyping and customization are desirable: an arm prototyped for a future Mars mission, and a wearable backpack-mounted arm. We then propose a novel method to minimize the number of joints in the mechanism while maintaining its ability to reach workspace task poses. We combine these methods into a framework in which we iteratively optimize and simplify task-specialized manipulator designs.