Project Scope
Robotic manipulation relates to generating intelligent motion and control mechanisms to move a robotic arm for useful object interactions. Robot manipulators are widely used in logistics scenarios to automate pick and place tasks, for instance within Amazon warehouses to serve order items from conveyor belts. The problem is challenging when the objects are not neatly separated and precise motions and forces are necessary.
This project will require the student to build on existing open-source codebases to control a robot arm (Ufactory xarm7) both in simulation and with the real robot in the lab. The arm is fitted with a two-fingered gripper. The project will need the student to develop algorithms and learned policies (reinforcement learning or imitation learning) to use the grippers to extract a motivating set of tool objects that are commonly found in medical support applications.
The Hardware
The robot has seven joints and a two fingered gripper. The project will need the use of existing open-source libraries that enable the communication of commands to the robot, making it move and apply forces.
The Simulation
A simulation is used to mimic the real world in order to think ahead before executing actions or to evaluate machine learning models. The project might need the use of simulators including Isaacsim, Mujoco, or Gazebo.
Manipulation Task
These objects are commonly found in surgical scenarios. The tools are placed in tightly fitted silicone placeholders. A precise point of grasp and forces can extract a tool from its mount, at which point it can be handed over to a surgeon requesting it. The project will involve learning robust motions that can autonomously solve this task.
The research will involve designing and evaluating different motion generation strategies to pick the objects
- motion and path planning
- diffusion policies (imitation learning)
- reinforcement learning
Minimal Required Skills
- Proficiency in Python or C++
Preferred Skills
- Experience with deep learning frameworks (e.g., PyTorch, Tensorflow, Jax)
- Experience with simulators (e.g., Gazebo, Mujoco, Isaac Sim)
- Experience with The Robot Operating System (ROS)
- Understanding of basic motion planning algorithms (e.g., PRM, RRT)