16-899 Hands: Design and Control for Dexterous Manipulation

Spring 2010
TuTh 3:00--4:20
GHC 4102

In this course, we will survey robotic hands and learn about the human hand with the goal of understanding hand design and control for dexterity. Questions to be explored include the following. Should robot hand kinematics be humanlike? What robotic sensors are available / practical, how do they measure up to sensors in the human hand, and what sensing capabilities are required for dexterous manipulation? What is a good benchmark suite of tasks for evaluating dexterous behavior? How do we design control algorithms for dexterous manipulation in the presence of uncertainty? What can we learn from human manipulation performance to improve robotic manipulation capability? This is a reading and project course. Students will be asked to present one or two short research papers of their own and to design and carry out a final project.

Instructor: Nancy Pollard

Email: nsp@cs.cmu.edu
Office: Newell-Simon Hall 4228b, Phone x8-1479

Week of Tue Thu

Jan 12 Robot Hands Overview

Slides

Robot Hands Design Issues

Slides

Jan 19 Case study: Stanford / JPL hand - Design for fingertip grasping and manipulation

Matthew T. Mason and J. Kenneth Salisbury, Robot Hands and the Mechanics of Manipulation, MIT Press 1985.

Slides

Case study: Robonaut hand - Design for teleoperation and tool use

Lovchik, C.S.; Diftler, M.A.; The Robonaut hand: a dexterous robot hand for space, ICRA 1999, Volume 2, 10-15 May 1999 Page(s):907 - 912

Martin, T.B.; Ambrose, R.O.; Diftler, M.A.; Platt, R., Jr.; Butzer, M.J.; Tactile gloves for autonomous grasping with the NASA/DARPA Robonaut, ICRA 2004, Volume 2, Apr 26-May 1, 2004 Page(s):1713 - 1718

Ambrose, R.O.; Savely, R.T.; Goza, S.M.; Strawser, P.; Diftler, M.A.; Spain, I.; Radford, N.; Mobile manipulation using NASA's Robonaut, ICRA 2004, Volume 2, Apr 26-May 1, 2004 Page(s):2104 - 2109 Vol.2

Slides

Jan 26 Case study: Dollar / Howe hand - Design for robustness and simplicity for grasping

Aaron M. Dollar and Robert D. Howe, The SDM Hand: A Highly Adaptive Compliant Grasper for Unstructured Environments, International Journal of Robotics Research, 2010 (in press).

Aaron M. Dollar, Leif Jentoft, Jason H. Gao, and Robert D. Howe, Contact Sensing and Grasping Performance of Compliant Hands, Autonomous Robots, vol. 28(1), pp. 65-75, 2010.

Case study: Cyberhand - Design for prosthetic applications

Zollo, L. Roccella, S. Guglielmelli, E. Carrozza, M.C. Dario, P., Biomechatronic Design and Control of an Anthropomorphic Artificial Hand for Prosthetic and Robotic Applications, IEEE Transactions on Mechatronics, 12(4), 418-429, Aug 2007.

Dellon, B.T.; Matsuoka, Y. Robotic Systems for Rehabilitation, Exoskeleton, and Prosthetics: Now and for the Future, IEEE Robotics and Automation Society Magazine, 14(1), pp.30-34, 2007.

Prosthetic hand design in the 16th century (short clip)

Feb 2 Field trip to motion capture lab - Shadow hand and prototype hand at CMU (meet in WeH 1334)

Frank Rothling, Robert Haschke, Jochen J. Steil, and Helge Ritter, Platform Portable Anthropomorphic Grasping with the Bielefeld 20-DOF Shadow and 9-DOF TUM Hand, IROS 2007.

Case study: ACT hand - utilizing humanlike kinematics, dynamics and control mechanisms

Vande Weghe, M.; Rogers, M.; Weissert, M.; Matsuoka, Y. The ACT Hand: Design of the Skeletal Structure, The Proceedings of the IEEE Intl. Conference on Robotics and Automation, pp.3375-3379, 2004.

Gialias, N. Matsuoka, Y., Muscle actuator design for the ACT Hand, The Proceedings of the IEEE Intl. Conference on Robotics and Automation, pp.3380-3385, 2004.

Chang, L. and Matsuoka, Y. A Kinematic Thumb Model for the ACT Hand, The Proceedings of the IEEE Intl. Conference on Robotics and Automation, pp.1000-1005, 2006.

Matsuoka, Y., Afshar, P., Oh, M. On the Design of Robotic Hands for Brain Machine Interface Neurosurgical Focus, 20(5):E3 p.1-9, 2006.

Balasubramanian, R. and Matsuoka, Y., Biological stiffness control strategies for the Anatomically Correct Testbed (ACT) hand. In Proceedings of the IEEE International Conference on Robotics and Automation, 2008, Page(s):737 - 742, May 2008.

Feb 9 CLASS CANCELLED Final Project Brainstorming

Feb 16 Human Hand Kinematics and Driving Mechanisms (Joints, Muscles, and Tendons)

Lynette A. Jones and Susan L. Lederman, Human Hand Function, Oxford University Press 2006. Chapter 2.

Paul W. Brand and Anne M. Hollister, Clinical Mechanics of the Hand, Mosby, Inc., 1999. Chapter 3,4,5.

Human Hand Sensing and Grasping

Lynette A. Jones and Susan L. Lederman, Human Hand Function, Oxford University Press 2006. Chapter 4,6,8

Feb 23 Evolution of the Hand for Grasping and Tool Use

Marzke, M.W. & Marzke, R.F. Evolution of the human hand: Approaches to acquiring, analyzing and interpreting the anatomical evidence. Journal of Anatomy, 197(Part 1), 121-140, 2000.

R. W. Young, Evolution of the human hand: the role of throwing and clubbing," Journal of Anatomy, 202, 165-174, 2003.

Human Grasp Control

Venkadesan M and Valero-Cuevas FJ. Neural Control of Motion-to-Force Transitions with the Fingertip. J. Neurosci., Feb 2008; 28: 1366 - 1373; doi:10.1523/JNEUROSCI.4993-07.2008.

De Gregorio, M., Bair, K., and Santos, V. J. Characterizing reflexive grip responses to rotational perturbations of an object in precision grasp. Proc Robotics: Science and Systems Conf, Seattle, WA, June 28 - July 1, 2009.

Mar 2 Object Recognition by Touch (Gurdayal Koonjul)

Klatzky, R. L., & Lederman, S. J. (2008). Object recognition by touch. In J. J. Rieser, D. Ashmead, F. Ebner, & A. Corn (Eds.), Blindness and brain plasticity in navigation and object perception (pp. 185-207). Mahwah, NJ: Erlbaum.

Grasping and Manipulation in Graphics (Khoon Kiat Ong)

C. Karen Liu, Dextrous Manipulation from a Grasping Pose, ACM Transactions on Graphics (SIGGRAPH) 2009

C. Karen Liu, Synthesis of Interactive Hand Manipulation, ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2008

Mar 9 Spring Break Spring Break

Mar 16 Design Optimization and Curved Fingers (Jong Eun Kim)

Aaron M. Dollar and Robert D. Howe, Towards Grasping in Unstructured Environments: Grasper Compliance and Configuration Optimization, Advanced Robotics, vol. 19(5), pp. 523-543, 2005.

Grasp Quality

Ravi Balasubramanian, Ling Xu, Peter Brook, Joshua R. Smith, Yoky Matsuoka, Human-Guided Grasp Measures Improve Grasp Robustness on a Physical Robot. To appear in Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA 2010).

Matei Ciocarlie and Peter Allen, Design and Analysis Tool for Underactuated Compliant Hands, IEEE/RSJ International Conference on Intelligent Robots and Systems, October 11-15, 2009 St. Louis, pp. 5234-5239.

Matei Ciocarlie, Hao Dang, Jamie Lukos, Marco Santello, Peter Allen, Functional Analysis of Finger Contact Locations during Grasping, Third Joint EUROHAPTICS Conference and symposium on haptic interfaces for virtual environment and teleoperator systems, March 18-20, 2009, Salt Lake City.

Miller, Andrew and Peter Allen Examples of 3-D Grasp Quality Computations, IEEE Int. Conf. on Robotics and Automation, May 10-15, 1999, Detroit, MI., pp. 1240-1246.

Y. Li, J. L. Fu, and N. S. Pollard, 2007. Data Driven Grasp Synthesis using Shape Matching and Task-Based Pruning, IEEE Transactions on Visualization and Computer Graphics, 13(4), 732--747, July/August 2007.

Kimmerle M, Mainwaring L, Borenstein M., The functional repertoire of the hand and its application to assessment. Am J Occup Ther. 2003 Sep-Oct;57(5):489-98.

Mar 23 Grasp Subspaces (Alberto Rodriguez)

Matei T. Ciocarlie and Peter K. Allen, Hand Posture Subspaces for Dexterous Robotic Grasping, The International Journal of Robotics Research 2009 28: 851-867.

GraspIt! website

Vision and Grasping I: Danica Kragic's group and Charles Kemp's group

B. Rasolzadeh, M. Björkman, K. Huebner, and D. Kragic, An Active Vision System for Detecting, Fixating and Manipulating Objects in Real World, The International Journal of Robotics Research first published on August 28, 2009

Advait Jain & Charles C. Kemp , EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces, Autonomous Robots, 28(1), pp. 45 - 64, 2010.

Mar 30 Final Project Updates Final Project Updates

Apr 6 Vision and Grasping II: Andrew Ng's group (Mehmet Dogar)

Ashutosh Saxena, Justin Driemeyer, and Andrew Y. Ng., Robotic Grasping of Novel Objects using Vision, In International Journal of Robotics Research (IJRR), 2008.

Ashutosh Saxena, Lawson Wong, and Andrew Y. Ng. Learning grasp strategies with partial shape information, In AAAI, 2008.

Ellen Klingbeil, Ashutosh Saxena, Andrew Y. Ng. Learning to Open New Doors, In Robotics Science and Systems (RSS) workshop on Robot Manipulation, 2008.

Kaijen Hsiao, Paul Nangeroni, Manfred Huber, Ashutosh Saxena and Andrew Y. Ng, Reactive Grasping using Optical Proximity Sensors, In International Conference on Robotics and Automation (ICRA), 2009.

Coaching Grasping

Ralph, M.; Moussa, M.A., Toward a Natural Language Interface for Transferring Grasping Skills to Robots, IEEE Transactions on Robotics,Volume: 24 Issue: 2 Date: April 2008, Page(s): 468-475

Apr 13 Manipulation of Flexible Objects (Ken Toh)

M. Bell, D. Balkcom. Grasping non-stretchable cloth polygons. International Journal of Robotics Research. Electronic publication August 2009

Yamakawa, Y.; Namiki, A.; Ishikawa, M.; Shimojo, M.; One-handed Knotting of a linear flexible object based on reconfigurable skill synthesis strategy, ASME/IFToMM International Conference on Reconfigurable Mechanisms and Robots (ReMAR 2009), 22-24 June 2009 Page(s):486 - 493

One-handed knotting website

NO CLASS (CARNIVAL)

Apr 20 Yamano / Maeno Ultrasonic Motor Hand (Tomoaki Mashimo)

Yamano, I.; Maeno, T. Five-fingered Robot Hand using Ultrasonic Motors and Elastic Elements, IEEE International Conference on Robotics and Automation (ICRA 2005) 18-22 April 2005 Page(s): 2673 - 2678.

Guest lecture on Preparatory Manipulation (Lillian Chang)
L. Y. Chang, S. S. Srinivasa, and N. S. Pollard, 2010. Planning pre-grasp manipulation for transport tasks, Proceedings of the IEEE International Conference on Robotics and Automation, Anchorage, Alaska, May 2010.

L. Y. Chang, R. L. Klatzky, and N. S. Pollard, 2010. Selection criteria for preparatory object rotation in manual lifting actions, Journal of Motor Behavior, 42(1):11--27, 2010.

L. Y. Chang, G. J. Zeglin, and N. S. Pollard, 2008. Preparatory object rotation as a human-inspired grasping strategy, IEEE International Conference on Humanoid Robots (Humanoids 2008), December 2008, pp 527--534.

L. Y. Chang and N. S. Pollard, 2009. Video survey of pre-grasp interactions in natural hand activities, Robotics: Science and Systems (RSS) 2009 Workshop on Understanding the Human Hand for Advancing Robotic Manipulation, June 2009. (poster)

Apr 27 Final Project Presentations Final Project Presentations

Week of	Tue	Thu
Jan 12	Robot Hands Overview Slides	Robot Hands Design Issues Slides
Jan 19	Case study: Stanford / JPL hand - Design for fingertip grasping and manipulation Matthew T. Mason and J. Kenneth Salisbury, Robot Hands and the Mechanics of Manipulation, MIT Press 1985. Slides	Case study: Robonaut hand - Design for teleoperation and tool use Lovchik, C.S.; Diftler, M.A.; The Robonaut hand: a dexterous robot hand for space, ICRA 1999, Volume 2, 10-15 May 1999 Page(s):907 - 912 Martin, T.B.; Ambrose, R.O.; Diftler, M.A.; Platt, R., Jr.; Butzer, M.J.; Tactile gloves for autonomous grasping with the NASA/DARPA Robonaut, ICRA 2004, Volume 2, Apr 26-May 1, 2004 Page(s):1713 - 1718 Ambrose, R.O.; Savely, R.T.; Goza, S.M.; Strawser, P.; Diftler, M.A.; Spain, I.; Radford, N.; Mobile manipulation using NASA's Robonaut, ICRA 2004, Volume 2, Apr 26-May 1, 2004 Page(s):2104 - 2109 Vol.2 Slides
Jan 26	Case study: Dollar / Howe hand - Design for robustness and simplicity for grasping Aaron M. Dollar and Robert D. Howe, The SDM Hand: A Highly Adaptive Compliant Grasper for Unstructured Environments, International Journal of Robotics Research, 2010 (in press). Aaron M. Dollar, Leif Jentoft, Jason H. Gao, and Robert D. Howe, Contact Sensing and Grasping Performance of Compliant Hands, Autonomous Robots, vol. 28(1), pp. 65-75, 2010.	Case study: Cyberhand - Design for prosthetic applications Zollo, L. Roccella, S. Guglielmelli, E. Carrozza, M.C. Dario, P., Biomechatronic Design and Control of an Anthropomorphic Artificial Hand for Prosthetic and Robotic Applications, IEEE Transactions on Mechatronics, 12(4), 418-429, Aug 2007. Dellon, B.T.; Matsuoka, Y. Robotic Systems for Rehabilitation, Exoskeleton, and Prosthetics: Now and for the Future, IEEE Robotics and Automation Society Magazine, 14(1), pp.30-34, 2007. Prosthetic hand design in the 16th century (short clip)
Feb 2	Field trip to motion capture lab - Shadow hand and prototype hand at CMU (meet in WeH 1334) Frank Rothling, Robert Haschke, Jochen J. Steil, and Helge Ritter, Platform Portable Anthropomorphic Grasping with the Bielefeld 20-DOF Shadow and 9-DOF TUM Hand, IROS 2007.	Case study: ACT hand - utilizing humanlike kinematics, dynamics and control mechanisms Vande Weghe, M.; Rogers, M.; Weissert, M.; Matsuoka, Y. The ACT Hand: Design of the Skeletal Structure, The Proceedings of the IEEE Intl. Conference on Robotics and Automation, pp.3375-3379, 2004. Gialias, N. Matsuoka, Y., Muscle actuator design for the ACT Hand, The Proceedings of the IEEE Intl. Conference on Robotics and Automation, pp.3380-3385, 2004. Chang, L. and Matsuoka, Y. A Kinematic Thumb Model for the ACT Hand, The Proceedings of the IEEE Intl. Conference on Robotics and Automation, pp.1000-1005, 2006. Matsuoka, Y., Afshar, P., Oh, M. On the Design of Robotic Hands for Brain Machine Interface Neurosurgical Focus, 20(5):E3 p.1-9, 2006. Balasubramanian, R. and Matsuoka, Y., Biological stiffness control strategies for the Anatomically Correct Testbed (ACT) hand. In Proceedings of the IEEE International Conference on Robotics and Automation, 2008, Page(s):737 - 742, May 2008.
Feb 9	CLASS CANCELLED	Final Project Brainstorming
Feb 16	Human Hand Kinematics and Driving Mechanisms (Joints, Muscles, and Tendons) Lynette A. Jones and Susan L. Lederman, Human Hand Function, Oxford University Press 2006. Chapter 2. Paul W. Brand and Anne M. Hollister, Clinical Mechanics of the Hand, Mosby, Inc., 1999. Chapter 3,4,5.	Human Hand Sensing and Grasping Lynette A. Jones and Susan L. Lederman, Human Hand Function, Oxford University Press 2006. Chapter 4,6,8
Feb 23	Evolution of the Hand for Grasping and Tool Use Marzke, M.W. & Marzke, R.F. Evolution of the human hand: Approaches to acquiring, analyzing and interpreting the anatomical evidence. Journal of Anatomy, 197(Part 1), 121-140, 2000. R. W. Young, Evolution of the human hand: the role of throwing and clubbing," Journal of Anatomy, 202, 165-174, 2003.	Human Grasp Control Venkadesan M and Valero-Cuevas FJ. Neural Control of Motion-to-Force Transitions with the Fingertip. J. Neurosci., Feb 2008; 28: 1366 - 1373; doi:10.1523/JNEUROSCI.4993-07.2008. De Gregorio, M., Bair, K., and Santos, V. J. Characterizing reflexive grip responses to rotational perturbations of an object in precision grasp. Proc Robotics: Science and Systems Conf, Seattle, WA, June 28 - July 1, 2009.
Mar 2	Object Recognition by Touch (Gurdayal Koonjul) Klatzky, R. L., & Lederman, S. J. (2008). Object recognition by touch. In J. J. Rieser, D. Ashmead, F. Ebner, & A. Corn (Eds.), Blindness and brain plasticity in navigation and object perception (pp. 185-207). Mahwah, NJ: Erlbaum.	Grasping and Manipulation in Graphics (Khoon Kiat Ong) C. Karen Liu, Dextrous Manipulation from a Grasping Pose, ACM Transactions on Graphics (SIGGRAPH) 2009 C. Karen Liu, Synthesis of Interactive Hand Manipulation, ACM SIGGRAPH/Eurographics Symposium on Computer Animation 2008
Mar 9	Spring Break	Spring Break
Mar 16	Design Optimization and Curved Fingers (Jong Eun Kim) Aaron M. Dollar and Robert D. Howe, Towards Grasping in Unstructured Environments: Grasper Compliance and Configuration Optimization, Advanced Robotics, vol. 19(5), pp. 523-543, 2005.	Grasp Quality Ravi Balasubramanian, Ling Xu, Peter Brook, Joshua R. Smith, Yoky Matsuoka, Human-Guided Grasp Measures Improve Grasp Robustness on a Physical Robot. To appear in Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA 2010). Matei Ciocarlie and Peter Allen, Design and Analysis Tool for Underactuated Compliant Hands, IEEE/RSJ International Conference on Intelligent Robots and Systems, October 11-15, 2009 St. Louis, pp. 5234-5239. Matei Ciocarlie, Hao Dang, Jamie Lukos, Marco Santello, Peter Allen, Functional Analysis of Finger Contact Locations during Grasping, Third Joint EUROHAPTICS Conference and symposium on haptic interfaces for virtual environment and teleoperator systems, March 18-20, 2009, Salt Lake City. Miller, Andrew and Peter Allen Examples of 3-D Grasp Quality Computations, IEEE Int. Conf. on Robotics and Automation, May 10-15, 1999, Detroit, MI., pp. 1240-1246. Y. Li, J. L. Fu, and N. S. Pollard, 2007. Data Driven Grasp Synthesis using Shape Matching and Task-Based Pruning, IEEE Transactions on Visualization and Computer Graphics, 13(4), 732--747, July/August 2007. Kimmerle M, Mainwaring L, Borenstein M., The functional repertoire of the hand and its application to assessment. Am J Occup Ther. 2003 Sep-Oct;57(5):489-98.
Mar 23	Grasp Subspaces (Alberto Rodriguez) Matei T. Ciocarlie and Peter K. Allen, Hand Posture Subspaces for Dexterous Robotic Grasping, The International Journal of Robotics Research 2009 28: 851-867. GraspIt! website	Vision and Grasping I: Danica Kragic's group and Charles Kemp's group B. Rasolzadeh, M. Björkman, K. Huebner, and D. Kragic, An Active Vision System for Detecting, Fixating and Manipulating Objects in Real World, The International Journal of Robotics Research first published on August 28, 2009 Advait Jain & Charles C. Kemp , EL-E: an assistive mobile manipulator that autonomously fetches objects from flat surfaces, Autonomous Robots, 28(1), pp. 45 - 64, 2010.
Mar 30	Final Project Updates	Final Project Updates
Apr 6	Vision and Grasping II: Andrew Ng's group (Mehmet Dogar) Ashutosh Saxena, Justin Driemeyer, and Andrew Y. Ng., Robotic Grasping of Novel Objects using Vision, In International Journal of Robotics Research (IJRR), 2008. Ashutosh Saxena, Lawson Wong, and Andrew Y. Ng. Learning grasp strategies with partial shape information, In AAAI, 2008. Ellen Klingbeil, Ashutosh Saxena, Andrew Y. Ng. Learning to Open New Doors, In Robotics Science and Systems (RSS) workshop on Robot Manipulation, 2008. Kaijen Hsiao, Paul Nangeroni, Manfred Huber, Ashutosh Saxena and Andrew Y. Ng, Reactive Grasping using Optical Proximity Sensors, In International Conference on Robotics and Automation (ICRA), 2009.	Coaching Grasping Ralph, M.; Moussa, M.A., Toward a Natural Language Interface for Transferring Grasping Skills to Robots, IEEE Transactions on Robotics,Volume: 24 Issue: 2 Date: April 2008, Page(s): 468-475
Apr 13	Manipulation of Flexible Objects (Ken Toh) M. Bell, D. Balkcom. Grasping non-stretchable cloth polygons. International Journal of Robotics Research. Electronic publication August 2009 Yamakawa, Y.; Namiki, A.; Ishikawa, M.; Shimojo, M.; One-handed Knotting of a linear flexible object based on reconfigurable skill synthesis strategy, ASME/IFToMM International Conference on Reconfigurable Mechanisms and Robots (ReMAR 2009), 22-24 June 2009 Page(s):486 - 493 One-handed knotting website	NO CLASS (CARNIVAL)
Apr 20	Yamano / Maeno Ultrasonic Motor Hand (Tomoaki Mashimo) Yamano, I.; Maeno, T. Five-fingered Robot Hand using Ultrasonic Motors and Elastic Elements, IEEE International Conference on Robotics and Automation (ICRA 2005) 18-22 April 2005 Page(s): 2673 - 2678.	Guest lecture on Preparatory Manipulation (Lillian Chang) L. Y. Chang, S. S. Srinivasa, and N. S. Pollard, 2010. Planning pre-grasp manipulation for transport tasks, Proceedings of the IEEE International Conference on Robotics and Automation, Anchorage, Alaska, May 2010. L. Y. Chang, R. L. Klatzky, and N. S. Pollard, 2010. Selection criteria for preparatory object rotation in manual lifting actions, Journal of Motor Behavior, 42(1):11--27, 2010. L. Y. Chang, G. J. Zeglin, and N. S. Pollard, 2008. Preparatory object rotation as a human-inspired grasping strategy, IEEE International Conference on Humanoid Robots (Humanoids 2008), December 2008, pp 527--534. L. Y. Chang and N. S. Pollard, 2009. Video survey of pre-grasp interactions in natural hand activities, Robotics: Science and Systems (RSS) 2009 Workshop on Understanding the Human Hand for Advancing Robotic Manipulation, June 2009. (poster)
Apr 27	Final Project Presentations	Final Project Presentations