The ability to manipulate deformable objects, such as textiles or paper, is a major prerequisite to bringing the capabilities of articulated robot hands closer to the level of manual intelligence exhibited by humans. We concentrate on the manipulation of paper, which affords us a rich interaction domain that has not yet been solved for anthropomorphic robot hands. Robust tracking and physically plausible modeling of the paper as well as feedback based robot control are crucial components for this task. This paper makes two novel contributions to this area. The first concerns real-time modeling and visual tracking. Our technique not only models the bending of a sheet of paper, but also paper crease lines which allows us to monitor deformations. The second contribution concerns enabling an anthropomorphic robot to fold paper, and is accomplished by introducing a set of tactile- and vision-based closed loop controllers.