Teleoperation w/ Predictive Display
We consider teleoperation of a platoon of distributed nonholonomic WMRs, each equipped only with onboard sensing/computing and p2p communication. We first utilize passive nonholonomic decomposition to split the platoon kinematics into formation-keeping and collective tele-driving, and design autonomous formation and teleoperation control fully incorporating nonholonomic constraint and distribution requirement. To address the estimation uncertainty and behavior complexity inherent to a large-scale platoon of distributed robots, we propose novel predictive display, which provides the user with the predicted future course of platoon motion with uncertainty (Autonomous Robots 2018)
Shared Peg-in-Hole Teleoperation of Stage-Arm System on Flexible Beam
We present experimental results of peg-in-hole teleoperatoin of a stage-manipulator system on vertical beam. Passivity-based control (IROS2017) is design to allow the robot to track the human command while suppressing beam vibration. User teleoperates the end-effector using haptic device with interaction force-feedback; HMD with 3D visual information via stereo camera, whose position is also perceptually optimized.
FPV Cooperative WMRs Teleoperation
We consider two nonholonomic WMRs cooperatively squeeze-grasping deformable object and teleoperated via FPV from the rear WMR. Nonholonomic passive decomposition is used to split WMRs' kinematics into grasping and teleoperation aspects. We then design semi-autonomous teleoperation consisting of: 1) autonomous grasping control; and 2) FPV camera teleoperation. Visuo-haptic feedback is designed to notify users of, and also nudge them to reduce, the command-behavior mismatch, which, being difficult to understand via (limited) FPV, thus can cause user confusion/frustration and performance deterioration it left undealt. (IJRR 2017)
Whole-Body Multi-Modal Teleoperation
Whole-body multi-modal semi-autonomous teleoperation framework for mobile manipulator systems, which consists of: 1) motion capture and whole-body motion and perception mapping; 2) slave robot autonomous control to real-time solve optimal inverse kinematics to track user command under user-slave kinematic dissimilarity; and 3) visuo-haptic-vestibular feedback with HMD, wearable haptic device for force feedback, and actuated chair for vestibular feedback to reduce HMD-induced motion sickness. (ICRA2015)
Vision-Based UAV/UGV Teleoperation
Vision-based multi-modal semi-autonomous teleoperation framework of a team of UAV/UGV is presented, where a human user tele-navigate the UAV/UGV team while seeing through front-view camera attached on the UAV, while the UAV/UGV coordination is maintained autonomous through downview camera attached on the UGV. Multi-modal feedback, consisting of FPV view for tele-navigation and haptic feeback to prevent UAV-UGV separation, is adopted to reduce sensory overload when the user relies on both the cameras at the same time. (ICRA2013, URAI2013 Best Video Award)
Multi-UAV Haptic Teleoperation
Semi-autonomous haptic teleoperation architecture is presented for multiple UAVs, consisting of: 1) UAV control layer, where each UAV is abstracted by, and is controlled to follow trajectory of, its own kinematic virtual point (VP); 2) VP control layer, which modulates each VP’s motion according to teleoperation commands and local artificial potentials (for VP–VP/VP-obstacle collision avoidance, VP–VP connectivity preservation); and 3) teleoperation layer, through which a single user can command all (or some) of VPs’ velocity with haptic feedback. (TMECH2013, Joint work with Max-Planck Institute, Germany)
Stable Bilateral Teleoperation
The issue of stability in bilateral (or force-reflecting) teleoperation is challenging, since it involves humans, that are in general impossible to model in a mathematically tractable way; and also imperfect communication (e.g., delay, packet loss, jitter, etc.), that is well-known for its destabilizing effect. In a line of research, we consider this problem of stable bilateral teleoperation: teleoperation of robots with general nonlinear dynamics (IEEE TRO2005); PD-based passive teleoperation with communication delay (IEEE TRO2006); and passive set-position modulation (PSPM) for arbitrary communication imperfectness (IEEE TRO2010).