TELEROBOTICS

Teleoperation of Platoon WMRs

We consider teleoperation of a platoon of distributed nonholonomic WMRs, each equipped only with onboard sensing/computing and p2p communication. We first utilize passive nonholonomic decomposition to split the platoon kinematics into formation-keeping and collective tele-driving, and design autonomous formation and teleoperation control fully incorporating nonholonomic constraint and distribution requirement. To address the estimation uncertainty and behavior complexity inherent to a large-scale platoon of distributed robots, we propose novel predictive display, which provides the user with the predicted future course of platoon motion with uncertainty (Autonomous Robots 2018)

We present experimental results of peg-in-hole teleoperatoin of a stage-manipulator system on vertical beam. Passivity-based control (IROS2017) is design to allow the robot to track the human command while suppressing beam vibration. User teleoperates the end-effector using haptic device with interaction force-feedback; HMD with 3D visual information via stereo camera, whose position is also perceptually optimized.

Shared Peg-in-Hole Teleoperation of Stage-Arm System on Flexible Beam

FPV Cooperative WMRs Teleoperation

We consider two nonholonomic WMRs cooperatively squeeze-grasping deformable object and teleoperated via FPV from the rear WMR. Nonholonomic passive decomposition is used to split WMRs' kinematics into grasping and teleoperation aspects. We then design semi-autonomous teleoperation consisting of: 1) autonomous grasping control; and 2) FPV camera teleoperation. Visuo-haptic feedback is designed to notify users of, and also nudge them to reduce, the command-behavior mismatch, which, being difficult to understand via (limited) FPV, thus can cause user confusion/frustration and performance deterioration it left undealt. (IJRR 2017)

Whole-Body Multi-Modal Teleoperation

Whole-body multi-modal semi-autonomous teleoperation framework for mobile manipulator systems, which consists of: 1) motion capture and whole-body motion and perception mapping; 2) slave robot autonomous control to real-time solve optimal inverse kinematics to track user command under user-slave kinematic dissimilarity; and 3) visuo-haptic-vestibular feedback with HMD, wearable haptic device for force feedback, and actuated chair for vestibular feedback to reduce HMD-induced motion sickness. (ICRA2015)

Vision-Based UAV/UGV Teleoperation

Vision-based multi-modal semi-autonomous teleoperation framework of a team of UAV/UGV is presented, where a human user tele-navigate the UAV/UGV team while seeing through front-view camera attached on the UAV, while the UAV/UGV coordination is maintained autonomous through downview camera attached on the UGV. Multi-modal feedback, consisting of FPV view for tele-navigation and haptic feeback to prevent UAV-UGV separation, is adopted to reduce sensory overload when the user relies on both the cameras at the same time. (ICRA2013, URAI2013 Best Video Award)

Semi-autonomous haptic teleoperation architecture is presented for multiple UAVs, consisting of: 1) UAV control layer, where each UAV is abstracted by, and is controlled to follow trajectory of, its own kinematic virtual point (VP); 2) VP control layer, which modulates each VP’s motion according to teleoperation commands and local artificial potentials (for VP–VP/VP-obstacle collision avoidance, VP–VP connectivity preservation); and 3) teleoperation layer, through which a single user can command all (or some) of VPs’ velocity with haptic feedback. (TMECH2013, Joint work with Max-Planck Institute, Germany)

Multi-UAV Haptic Teleoperation

Passive Bilateral Teleoperation

The issue of stability in bilateral (or force-reflecting) teleoperation is challenging, since it involves humans, that are in general impossible to model in a mathematically tractable way; and also imperfect communication (e.g., delay, packet loss, jitter, etc.), that is well-known for its destabilizing effect. In a line of research, we consider this problem of stable bilateral teleoperation: teleoperation of robots with general nonlinear dynamics (IEEE TRO2005); PD-based passive teleoperation with communication delay (IEEE TRO2006); and passive set-position modulation (PSPM) for arbitrary communication imperfectness (IEEE TRO2010).

INRoL