16-899 Hands: Design and Control for Dexterous Manipulation

Research related to hands has increased dramatically over the past decade. Hands are in focus in computer graphics and virtual reality, new robot hands have been popping up in great variety, and manipulation has been featured in widely publicized programs such as the DARPA Robotics Challenge. With all of this attention on hands, are we close to a breakthrough in dexterity, or are we still missing some things needed for truly competent manipulation?

In this course, we will survey robotic hands and learn about the human hand with the goal of pushing the frontiers on hand design and control for dexterous manipulation. We will consider the necessary kinematics and dynamics for dexterity, what sensors are required to carry out dexterous interactions, the importance of reflexes and compliance, and the challenge of uncertainty. We will examine the human hand: its structure, sensing capabilities, human grasp choice and control strategies for inspiration and benchmarking. Students will be asked to present one or two research papers, participate in discussions and short research or design exercises, and carry out a final project.

Instructor: Nancy Pollard

• Email: nsp@cs.cmu.edu

• Office: EDSH 227, Phone x8-1479

Syllabus

The syllabus is subject to change, but here is a draft:

Week of	Mon	Wed
Jan 11	Introduction Slides	Human Grasping Slides Reference List
Jan 18	NO CLASS - Martin Luther King Day	More on Human Grasping Slides Reference List
Jan 25	Mathematical Foundations Reference List	Case Study: Stanford/JPL Hand Reference List
Feb 1	Gripper Optimization Reference List	Learning Manipulation Skills Reference List
Feb 8	Yuzi's Experiment	Manipulation Policies (Michael Koval) Slides
Feb 15	Manipulation / Underactuated Hands (Robbie) Reference List	Manipulation Planning Reference List
Feb 22	Igor Mordatch Job Talk 1PM in NSH 3305	Taxonomy Followup (Rosario) Reference List
Feb 29	FINAL PROJECT PITCHES	FINAL PROJECT PITCHES
Mar 7	SPRING BREAK	SPRING BREAK
Mar 14	Compliant Joints / Hands (Reuben)	Compliance in the Human Hand Reference List
Mar 21	Soft Hands (Gilwoo)	Robot Skin and Tactile Sensing (David)
Mar 28	CANCELLED	Grasp Planning / Shape (Puneet)
Apr 4	Grasp Quality (Jonathan)	More Grasp Quality Reference List
Apr 11	Human Hand Anatomy / Capabilities Slides Reference List	Object Factors -> Grasp Choice (Yuzi)
Apr 18	Grasp Classification / Muscle Signals (Se-Joon)	Human Grasp / Manipulation Follow-up
Apr 25	Final Project Presentations	Final Project Presentations

Topics of interest for this course include:

• ROBOT HANDS
  We will study existing robot hands and their properties. We'll look at soft hands, underactuated hands, hands utilizing synergies, highly dexterous hands, prosthetic hands, universal grippers, and everything else we can uncover. We will compare these hands along dimensions of strength, weight, sensing capability, ability to securely grasp, ability to manipulate, and aspects which make each hand unique. One outcome of the course will be a document or wiki summarizing information uncovered in this survey.

• HUMAN HANDS
  We will learn about human hand kinematic and dynamic properties and the internal structure of the hand, including our many sensors. We will consider hand evolution, review what is known about how we control our hands, look at where human hand simulations succeed and fail, and evaluate the human hand along the same guidelines as we evaluate the various robotic hands to uncover obvious gaps and consider their importance.

• HUMAN GRASPING IN THE WILD
  How many grasps? How many manipulation actions?
  Many taxonomies of human grasping have been proposed, and they seem to be getting more detailed and complex with the passage of time, due to evident variety in hand shape, motion, force, stiffness, and intent. However, there are some clear organizing principles and strong similarities across classes of grasps and manipulation actions. We will attempt to identify the few canonical grasp varieties that are most important to human grasping in the wild and understand how to adapt and move between them -- in other words we will attempt to write a complete playbook for everyday grasping and manipulation.

• HAND SHAPE AND JOINT LIMITS
  The shape of a robot finger, and the "negative space" between fingers and palm as the hand closes can strongly influence grasp success. Joint limits are also very important, yet the role of joint limits in optimizing stability and force delivery has been little (not at all?) studied. We will explore both of these aspects of robot hand design.

• COMPLIANCE AND REFLEXES
  I believe two of the reasons for success in human grasping are passive compliance and reflexes, resulting in very fast, carefully shaped responses to contact and collision. We will study what is known about reflexes and recent research results on human finger compliance, look at some real world examples, and see if research relating to series elastic actuators and other compliant design elements aligns with what we think we want in a grasp / manipulation controller.

• UNCERTAINTY
  We will look into the literature on learning, optimization, and planning that is relevant to grasping and manipulation in the cluttered and uncertain real world. One particular question we will consider is how data captured from real world interactions can improve our ability to handle and plan for uncertainty.

• QUALITY AND METRICS
  How do we analyze grasps and what makes for a high quality grasp? There is an established mathematics for grasp analysis, and there are more than a dozen measures out there, but do any of them capture our intuition for what we want a grasp to be? Are any of them good enough to be used as the basis for grasp optimization? Grasping is not just a final result, but a process that may involve some manipulation in order to acquire the object. How should we evaluate a manipulation plan or policy? We will look at various quality measures and objectives that have been posed and stack them up against our intuition for what we really want as an objective.