16-848 Hands: Design and Control for Dexterous Manipulation

Spring 2020
MW 1:30-2:50pm
GHC 4102

Research related to hands has increased dramatically over the past decade. Robot hand innovation may be at an all time high, with new materials and manufacturing techniques promoting an explosion of ideas. Hands have become a priority in virtual reality and telepresence. Even the study of how people use their hands is seeing the growth of new ideas and themes.

With all of this attention on hands, are we close to a breakthrough in dexterity, or are we still missing some things needed for truly dexterous manipulation?

In this course, we will survey robotic hands and learn about the human hand with the goal of pushing the frontiers on hand design and control for dexterous manipulation. We will consider the necessary kinematics and dynamics for dexterity, what sensors are required to carry out dexterous interactions, the importance of reflexes and compliance, the role of machine learning in grasping and manipulation, and the challenge of uncertainty. We will explore state of the art manufacturing and design techniques, including innovations in soft robotics and embedded sensing. We will examine the human hand: its structure, sensing capabilities, human grasp choice and control strategies for inspiration and benchmarking. Students will be asked to present one or two research papers, participate in discussions and short research or design exercises, and carry out a final project.

Instructor: Nancy Pollard


Syllabus

Week of Mon Wed
Jan 13 Course Introduction / Historical Robot Hands More Intro / Case Study: JamHand
Jan 20 No Class: Martin Luther King Day
Course Introduction / The Human Hand
Jan 27 Case Study: DLR Variable Stiffness Hands Final Project Overview / Brainstorming
Feb 3 Grasp Taxonomies for Humans and Robots (Avi) More on Taxonomies / Grasping in the Wild
Feb 10 Synergies in the Pisa/IIT SoftHand (Ryan) More on Synergies
Feb 17 Grasp Quality Benchmarks, Datasets, and Measures of Dexterity
Feb 24 Laschi Underactuated Soft Gripper Case Study (Sarah) Foam Hands Case Study (Dominik Bauer)
Mar 2 Developments in Soft Prosthesis Design (Keene) FINAL PROJECT PITCHES
Mar 9 SPRING BREAK
SPRING BREAK
Mar 16 CLASS CANCELLED
Soft Sensor Skin (E-Skin) (Jonathan)
Mar 23 Tactile Sensing Touch-based Grasp Primitives (Paulo)
Mar 30 Touch Based Control Manipulation Planning
Apr 6 FINAL PROJECT CHECKPOINT
Modeling Uncertainties of Dynamics and Interaction (Caroline)
Apr 13 More on Uncertainties Learning Manipulation I
Apr 17 (Friday) Learning Manipulation II
Apr 20 Design Optimization I Design Optimization II
Apr 27 FINAL PROJECT PRESENTATIONS
FINAL PROJECT PRESENTATIONS


Topics of interest for this course include:


Grading Information

Grading for the class will be as follows:


Paper Presentation

You will be required to present one recent research paper during the course of the semester. You will have the entire class period available to you, but you are not required to use all of this time. A typical presentation time may be 45 to 60 minutes. (Less than 30 minutes is too short.)

In your presentatation, please cover the points listed below. In general, plan for about one minute per slide. Make sure the text is easy to read, and replace text with images, plots, videos, and diagrams where it helps with explanation. You will be asked to turn in your presentation material by sending it in an email to me either before or after your presentation.

My goal is that the class can learn, think about, and discuss the material in the paper you choose, and you will be graded based on clarity of the presentation and on providing points that can raise discussion in the class. (Don't worry, I will take into account that different papers are more and less easy to present clearly and more and less easy to discuss!)

Here are the points to cover. You don't have to follow this order, but most papers do. If you have a collection of papers or a survey, your approach will be slightly different.


Final Project

Overview:
In this assignment, you can work individually or in a group of 2-3 to implement a project you are interested in. This project should be related to some aspect of hands and/or dexterous manipulation. You can choose from the topics covered in our class, or you can find another topic of your choice. Clear all topics with me early on. I can also help you find resources and choose an appropriate scope.

Deadlines:

Guidelines:

Meeting. Sometime before the final project pitches, every group should schedule a meeting with me to discuss final project ideas. Please do this even if we have already discussed your project in the hallway or after class. Bring to the meeting your ideas, preliminary research, thoughts about resources, scope, concerns, etc. If you are not sure about what you want to do yet, try to come to the meeting with three possible proposal ideas. Even if they are very unformed, we can work on them in the meeting.

Final Project Pitches. The goal of final project pitches is to introduce to the rest of the class your final project ideas, get feedback and suggestions related to resources, and make any last decisions before submitting your proposal. Ideas should be fairly far along by this point, although you may be deciding between two possible final project ideas. This presentation can be informal. Visuals are always helpful, but they are not required. Final project pitches have a time limit of 10 minutes, but you may or may not need all of that time, depending on the stage of your project. Plan to leave at least two minutes for feedback and questions. Although it is informal, do tell us what you are thinking of doing, why you chose that topic, what resources you have available to you, how you will measure progress / success, and what you are thinking of in terms of goals / milestones.

Written Proposal. The written proposal is your first formal deliverable. Please submit it by email to nsp at cs.cmu.edu by 11:59pm on the day of the deadline. Your proposal should be a written document of approximately 2-3 pages, and should contain the following sections:

Final Project Presentation. Your final project presentation should be a fairly formal presentation, with slides and results. You should have 20-25 minutes available to you for this presentation. Your presentation should review your motivation and project goals, your approach, and evaluation plan, and give results. You should discuss your results and challenges you met along the way, things you learned while doing this project, and what you would do if you had more time. This should be a presentation version of your final report, which will be described next.

Final Report. Your final report can be an updated version of the proposal. It should be a document of approximately 6-10 pages. Please hand in both your written report and final project presentation by emailing them to nsp at cs dot cmu dot edu.

  • You should correct and expand the technical section of the proposal and describe the algorithms, designs, etc. that you actually use in your project.
  • Present some results (plots, tables, screen shots, images, videos...), making use of the evaluation metrics outlined in your proposal.
  • Discuss the results. Describe pros and cons of your approach.
  • State the problems you encountered and how you solved them.
  • Tell us what else you would do or what would you do differently if you had time.