16-848 Hands: Design and Control for Dexterous Manipulation

Spring 2024
MW 12:30-1:50pm

Research related to hands has increased dramatically over the past decade. Robot hand innovation may be at an all time high, with new materials and manufacturing techniques promoting an explosion of ideas. Hands have become a priority in virtual reality and telepresence. Even the study of how people use their hands is seeing the growth of new ideas and themes.

With all of this attention on hands, are we close to a breakthrough in dexterity, or are we still missing some things needed for truly dexterous manipulation?

In this course, we will survey robotic hands and learn about the human hand with the goal of pushing the frontiers on hand design and control for dexterous manipulation. We will consider the necessary kinematics and dynamics for dexterity, what sensors are required to carry out dexterous interactions, the importance of reflexes and compliance, the role of machine learning in grasping and manipulation, and the challenge of uncertainty. We will explore state of the art manufacturing and design techniques, including innovations in soft robotics and embedded sensing. We will examine the human hand: its structure, sensing capabilities, human grasp choice and control strategies for inspiration and benchmarking. Students will be asked to present one or two research papers, participate in discussions and short research or design exercises, and carry out a final project.

Instructor: Nancy Pollard

Piazza: We will use piazza for announcements and class discussion. Our piazza link is https://piazza.com/class/lrhwmvk3t7v1eq/


Week of Mon Wed
Jan 15 NO CLASS: Martin Luther King Day Course Introduction
Jan 22 Some History of Robot Hands Developing a Dexterous Soft Hand (Pragna Mannam)
Jan 29 The Human Hand Taxonomies, Synergies, and Benchmarks
Feb 5 Final Project Brainstorming FFLEX Hand V1 & V2 (Zilin Si)
Feb 12 Third Thumb (Danny Magruder)
Mocap Lab Demo (WeH 1334)
Feb 19 Final Project Pitches
The Palm (Sandhya Sridhar)
Feb 26 Dexterous Soft Hand Design (Dominik Bauer)
Direct Drive Hands (Quanting Xie)
Hand Tracking and Simulation in VR (Edward Lu)
Mar 18 DexPilot (Sruti Srinidhi)
Hand Mapping for Teleoperation (Madeleine Weaver)
Mar 25 Final Project Checkpoint
Topics on Prosthetic Hands
Apr 1 GelSight EndoFlex (Jean Udompanyawit)
Hands and Sensing
Apr 8 Math for Grasping and Manipulation
More on Force Sensing and Haptic Feedback
Apr 15 Active Surfaces for Manipulation (Jinyun Xu)
Design Optimization
Apr 22 Learning for Dexterity
Hands in Graphics / Hand Control

Topics of interest for this course include:

Grading Information

Grading for the class will be as follows:

Paper Presentation

You will be required to present one recent research paper during the course of the semester. You will have the entire class period available to you, but you are not required to use all of this time. A typical presentation time may be 45 to 60 minutes. (Less than 30 minutes is too short.)

In your presentatation, please cover the points listed below. In general, plan for about one minute per slide. Make sure the text is easy to read, and replace text with images, plots, videos, and diagrams where it helps with explanation. You will be asked to turn in your presentation material by sending it in an email to me either before or after your presentation.

My goal is that the class can learn, think about, and discuss the material in the paper you choose, and you will be graded based on clarity of the presentation and on providing points that can raise discussion in the class. (Don't worry, I will take into account that different papers are more and less easy to present clearly and more and less easy to discuss!)

Here are the points to cover. You don't have to follow this order, but most papers do. If you have a collection of papers or a survey, your approach will be slightly different.

Final Project

In this assignment, you can work individually or in a group of 2-3 to implement a project you are interested in. This project should be related to some aspect of hands and/or dexterous manipulation. You can choose from the topics covered in our class, or you can find another topic of your choice. Clear all topics with me early on. I can also help you find resources and choose an appropriate scope.



Meeting. Sometime before the final project pitches, every group should schedule a meeting with me to discuss final project ideas. Please do this even if we have already discussed your project in the hallway or after class. Bring to the meeting your ideas, preliminary research, thoughts about resources, scope, concerns, etc. If you are not sure about what you want to do yet, try to come to the meeting with three possible proposal ideas. Even if they are very unformed, we can work on them in the meeting.

Final Project Pitches. The goal of final project pitches is to introduce to the rest of the class your final project ideas, get feedback and suggestions related to resources, and make any last decisions before submitting your proposal. Ideas should be fairly far along by this point, although you may be deciding between two possible final project ideas. This presentation can be informal. Visuals are always helpful, but they are not required. Final project pitches have a time limit of 10 minutes, but you may or may not need all of that time, depending on the stage of your project. Plan to leave at least two minutes for feedback and questions. Although it is informal, do tell us what you are thinking of doing, why you chose that topic, what resources you have available to you, how you will measure progress / success, and what you are thinking of in terms of goals / milestones.

Written Proposal. The written proposal is your first formal deliverable. Please submit it by email to nsp at cs.cmu.edu by 11:59pm on the day of the deadline. Your proposal should be a written document of approximately 2-3 pages, and should contain the following sections:

Final Project Presentation. Your final project presentation should be a fairly formal presentation, with slides and results. You should have 20-25 minutes available to you for this presentation. Your presentation should review your motivation and project goals, your approach, and evaluation plan, and give results. You should discuss your results and challenges you met along the way, things you learned while doing this project, and what you would do if you had more time. This should be a presentation version of your final report, which will be described next.

Final Report. Your final report can be an updated version of the proposal. It should be a document of approximately 6-10 pages. Please hand in both your written report and final project presentation by emailing them to nsp at cs dot cmu dot edu.

  • You should correct and expand the technical section of the proposal and describe the algorithms, designs, etc. that you actually use in your project.
  • Present some results (plots, tables, screen shots, images, videos...), making use of the evaluation metrics outlined in your proposal.
  • Discuss the results. Describe pros and cons of your approach.
  • State the problems you encountered and how you solved them.
  • Tell us what else you would do or what would you do differently if you had time.