Research

Robotics: Medical Robotics, Haptics, Teleoperation, Design of Mechatronic Systems

Human-Machine Interfaces / Virtual Environments: Surgical Simulation, Haptic Interfacing to Virtual
Environments, Physical Modeling for Simulation in Virtual Environments

Systems and Control: Modeling and Simulation of Complex Biological Systems

Here is the list of my publications and my students (where you can find their theses as well).

For a list of my current research projects, please visit the web site of my research group:

Medical Robotics and Computer Integrates Surgery (MeRCIS) Laboratory.

Below, you can find highlights from my recent and past research projects.

HIGHLIGHTS FROM RECENT RESEARCH PROJECTS

Haptics and Telerobotics
Computational Modeling of Human Performance in Haptic Manipulation
Design of Bilateral Teleoperation Systems
Haptic Interfacing to Virtual Environments

Virtual Environment-Based Surgical Simulation
GiPSi Software Framework for Surgical Simulation
Physical Modeling for Simulation in Virtual Environments

Modeling and Simulation of Biological Systems
Phy-SIM: Physiological Simulation, Integration and Modeling Toolkit

Emergence of systems biology provided a comprehensive and integrative perspective to examine the function at the system level instead of focusing on the isolated parts. Integration of multilevel and multiscale physiological models is an important requirement for such a system-based approach. Mathematical models for physiological processes have been developed in all levels from cell, up to organs and organ systems. However, little has been done in the name of integrating individual models to comprehensively study the whole system. This is due to the complexity to integrate multiscale and multilevel models of independent physiological processes.

Phy-SIM aims to build a software framework where the multilevel and multiscale models for the complex biological system can be integrated. Phy-SIM provides the software tools to model, integrate and simulate models of physiological processes from tissue level up to organism level.

For more information please visit project page.

Our research focuses on development of algorithms for real-time simulation of soft tissue behavior and high fidelity haptic interfacing to deformable object models simulated in virtual environments, to generate a realistic environments for surgical training.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

GiPSi is an open source/open architecture framework for developing organ level surgical simulations. Our goal is to facilitate shared development of reusable models, to accommodate heterogeneous models of computation, and to provide a framework for interfacing multiple heterogeneous models. The framework provides an intuitive API for interfacing dynamic models defined over spatial domains. It is specifically designed to be independent of the specifics of the modeling methods used and therefore facilitates seamless integration of heterogeneous models and processes. I/O interfaces for visualization and haptics for real-time interactive applications have also been provided.

We have used the GiPSi framework in house for the development of a virtual neuro-endoscopy training simulator.

For more information please visit project page.

Haptic interaction is an increasingly common form of interaction in virtual environment simulations. As part of this research, we have developed of algorithms for high fidelity haptic rendering of sliding frictional contact during manipulation of deformable objects in virtual environments. In order to model intermittent and sliding contacts, we have developed a multi-rate simulation approach which employs a local geometric model a local dynamic model that are simulated at haptic update rates. The local geometric and dynamic models are used for performing collision detection and response as part of the high update rate haptic loop, with sub-sample collision detection and a novel constraint-based hybrid collision response method.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

The mechanism and control design of a robotic telesurgery system needs to be done in an integrated fashion to achieve highest fidelity telemanipulation, which is critical since surgery involves very fine and delicate manipulations. However, a quantitative framework that can be used to systematically analyze and evaluate haptic systems in application-critical tasks is not available.

As part of this research, we have focused on development of such a framework that will consider the kinematic, dynamic, actuation, and sensory aspects of the system together with the controller design to optimize the performance of the system with respect to application-based performance criteria and human sensory-motor abilities. We have developed a novel quantitative comparison method for haptic systems based on mu-synthesis framework. The method uses the reciprocal of the structured singular value of the system, including the H∞ sub-optimal controller designed using mu-synthesis technique with the proper uncertainty and performance blocks, as the quantitative index for objective comparison. The method is capable of handling operator, environment, and system model uncertainties, sensor noise, and human sensory-motor abilities.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

Haptics is a critical form of human interface in interacting with real or virtual environments, as it is the only active sense that can be used in exploring or experiencing an environment. As the user interacts with environments through the haptic system, it alters the user’s perception and motor control, which can affect task performance. Therefore, understanding a haptic system’s effects on the sensory-motor system and the implications of these interactions on task performance is important for the design of effective haptic interface systems.

Our research focuses on characterization, modeling, and analysis of human motor performance in the context of stylus-based haptic interface devices, combining human psychophysics experiments with analysis methods from system theory to model and study different aspects of human haptic interaction.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

The objective of this research is the development of a framework for assessing the reliability and safety of robotic surgery systems, including its hardware and software, during development, field testing, and general deployment. The framework will use accurate simulations to assess pre-clinical reliability before deployment. After deployment, the framework will use data collection through online monitoring of the system as it is being used in the field, followed by analysis to obtain assessments of operational reliability and safety. The framework also aims to support post-market surveillance of these systems for reassessing reliability and safety after system maintenance.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

Needle-based percutaneous interventions are widely performed for diagnostic and therapeutic purposes. In such interventions, needle placement accuracy is frequently cited as an important factor in determining the overall outcome the procedure. However, consistently accurate manual needle placement for targets embedded deep inside the tissue is a significant challenge, even under intra-operative image guidance. The robotic tools that will be developed in this project will overcome these difficulties, improving precision and consistency of percutaneous interventions, to reliably perform diagnostic interventions on smaller lesions and to effectively target them for therapy.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

Design of the robotic mechanisms for minimally invasive surgery is one of the thrust areas of our research. In addition to the development of a tele-surgical robotic system for beating heart surgery (described above), our research also focuses on development of miniaturized actuation and sensor systems that can be used as enabling technologies for robotic systems for minimally invasive surgery.

Examples include, an integrated hybrid actuator system that combines shape memory alloy and DC motor actuators, and a whisker sensor to detect the relative motion between the surgical instrument and the heart.

For more information please visit the MeRICS Laboratory web pages, and our related publications.

This project aims to develop models, algorithms, and testbeds for robust intelligent manipulation that will enable supervisory control for intelligent robotic surgical assistants (RSAs). The goal of this research is to make surgical robotic systems to act more like an assistant and less like a slave, such that, the surgeon will have a high-level interaction with the system rather than controlling it through low-level direct teleoperation. Specifically, our research has been focusing on development of robotic sensing, planning, and motion control algorithms for autonomously performing surgical suturing and retraction tasks.

For more information please visit MeRCIS Laboratory web pages, and our related publications.

This project focuses on development of intelligent robotic tools for performing off-pump (beating heart) coronary artery bypass graft (CABG) surgery. In this context, off-pump CABG surgery means that the surgery is done while the heart is still beating instead of using a cardiopulmonary bypass machine and stopping the heart to perform heart surgery. The intelligent telerobotic tools that are developed in this project will actively track and cancel the relative motion between the surgical instruments and the heart by Active Relative Motion Canceling (ARMC), allowing CABG surgeries to be performed on a beating heart with technical perfection equal to traditional on pump procedures.

For more information please visit project page.

Undergraduate Research

My graduation project at METU was "Closed Loop Position and Force Control of Anthrobot III Robot Hand," which was supported by Ankara Electronics Research and Development Institute (now BILTEN) of Turkish Scientific and Technical Research Council, under project number 95-20-100. My project advisor was Assoc. Prof. Dr. Aydan Erkmen.

HYBRID SYSTEMS

Force control and manipulation involving contacts are essentially hybrid control problems because of the inherent switching present in the dynamic behavior when the manipulator comes in contact with and leaves a surface. In this study, I used the game theoretic approach of hybrid control design is used to synthesize the least restrictive control law for a robotic manipulator to establish and maintain contact with a surface while keeping interaction forces within specified bounds.

Please refer to my publications for further information.

VIRTUAL ENVIRONMENTS FOR SURGICAL SIMULATION

Learning laparoscopic techniques is much more difficult for surgeons than learning open surgery procedures. Currently, surgeons are trained during actual operations or in the animal laboratory. Training in the operating room increases risk to the patient and slows the operation, resulting in greater cost. Animal training is expensive and cannot duplicate human anatomy. Computer-based training has many potential advantages. It is interactive, yet an instructor's presence is not necessary, so students may practice in their free moments. Any pathology or anatomical variation can be created. Simulated positions and forces can be recorded to compare with established performance metrics for assessment and credentialing. Students could also try different techniques and look at anatomy from perspectives that would be impossible during surgery.

In the context of this application, I have worked on the development of real-time finite element models of soft tissue behavior and high fidelity haptic interfacing to deformable object models simulated in virtual environments, to generate a realistic environment for training.

Please refer to Frank Tendick's Virtual Environments for Surgical Training and Augmentation (VESTA) page and my related publications for related information.

Haptic Interfacing to Virtual Environments

Haptic interaction is an increasingly common form of interaction in virtual environment (VE) simula-
tions. This medium introduces some new challenges. As part of my research, I have introduced a multirate simulation scheme which uses a local linear approximation to address the problem arising from the difference between the sampling rate requirements of haptic interfaces and the significantly lower update rates of the physical models being manipulated in surgical simulators.

Please refer to my publications for further information.

Open Source Modeling Framework for Biocomputation

The objective of this effort was to study organ level modeling and simulation for surgical applications. The point to the proposed research is to explore the feasibility of developing open source, open architecture models of different levels of granularity and spatio-temporal scale for a project that has been labeled the Digital Human project. While the emphasis on this program was on how the simulations that we develop will allow for the interconnections between individual organ simulations, and between different types of physical processes within a given organ, we developed our tools on a specific test bed application: the construction of a heart model for simulation of heart surgery.

This effort later evolved into the development of the GiPSi framework.

Development of an Echography Simulator with Force Feedback

While visiting the SHARP Group at the INRIA Rhone-Alpes Research Center in Grenoble, France, I worked on modeling the dynamics of human-thigh for development of an echography simulator with force feedback. We have developed a mass-spring model of the dynamics of a human thigh based on real data acquired. Using a force sensor mounted on a robot arm the deformation of the thigh with respect to an external force is measured. The stress-strain curves we obtained exhibit a strong non-linearity due to the incompressibility of the human tissue. Hence, we
propose a two-layer model of the thigh using both linear and non-linear visco-elastic springs to simulate the observed behaviour. The parameters of the springs are estimated using a least-squares minimisation method.

For further information, please refer to my publications.

BILATERAL TELEOPERATION

Minimally invasive surgery (MIS) can be of great benefit to the patient, but places great demands on the surgeon's perceptual motor skills. Teleoperation technology can restore some of the lost dexterity and sensation in MIS.

In this research project we studied teleoperation controller design for haptic exploration and telemanipulation of soft environments. Our research had three components: (a) experiments to determine human capability to discriminate changes in compliance displayed through a haptic interface, (b) analysis and design of teleoperator control algorithms to optimize the transmission of compliance, (c) experiments to evaluate operator performance using teleoperation systems in a task more representative of surgery, complementing the control design procedure. The paradigm used in all the cases is the ability to detect a change in compliance of a surface, as would occur due to a lesion or vessel embedded in soft tissue.

As part of this research project, I have proposed a new measure for fidelity in teleoperation which quantifies the teleoperation system's ability to transmit changes in the compliance of the environment. This sensitivity function also incorporates the experimentally measured frequency dependent compliance discrimination sensitivity of the human operator. The bilateral teleoperation controller design problem was then formulated in a task-based optimization framework as the optimization of this metric with constraints on free space tracking and robust stability of the system under environment and human operator uncertainties. We have also used this analysis to evaluate the effectiveness of using a force sensor in a teleoperation system.

Please refer to my related publications, Frank Tendick's web pages, and the UC Berkeley Medical Robotics Group web pages for more information.

ROBOTIC TELESURGICAL SYSTEM FOR LAPAROSCOPY

Research Projects During Graduate and Postdoctoral Studies at UC Berkeley

My PhD thesis at UC Berkeley was "Telesurgery and Surgical Simulation: Design, Modeling, and Evaluation of Haptic Interfaces to Real and Virtual Surgical Environments." My PhD advisor was Prof. S. Shankar Sastry, and my co-advisor was Prof. Frank Tendick. After my PhD, I stayed as a postdoctoral researcher / specialist with the Medical Robotics group at UC Berkeley, Dept. of EECS. During my PhD and postdoctoral studies, I worked on the following projects:

Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. Unfortunately, there are disadvantages due to the reduced dexterity, workspace, and sensory input to the surgeon which is only available through a single video image.

In this joint project between the Robotics and Intelligent Machines Laboratory of the University of California, Berkeley (UCB) and the Department of Surgery of the University of California San Francisco (UCSF), a robotic telesurgical workstation for laparoscop was developed. Our Robotic Telesurgical Workstation for Laparoscopy was a bimanual system with two 6 DOF manipulators instrumented with grippers, controlled by a pair of 6 DOF master manipulators.

With the telesurgical workstation, the conventional surgical tools are replaced with robotic instruments which are under direct control of the surgeon through teleoperation. The goal is to restore the manipulation and sensation capabilities of the surgeon which were lost due to minimally invasive surgery. A 6 DOF slave manipulator, controlled through a spatially consistent and intuitive master, will restore the dexterity, the force feedback to the master will increase the fidelity of the manipulation, and the tactile feedback will restore the lost tactile sensation.

Here you can find a video of the system in action (high resolution video [21MB], low resolution video [4MB]). You will need a DivX codec to view the videos.

My main roles in this project were the analysis of the manipulator, control design, tissue modeling, and evaluation of the system.

For more information please visit the UC Berkeley Medical Robotics Group web pages, and my related publications.

Workspace Analysis of Robotic Manipulators

As part of this project, I have also developed a method to evaluate the kinematic ability of surgical robotic manipulators to perform the critical tasks of suturing and knot tying. The method uses open (i.e., non-MIS) surgical suturing motion data collected from experiments done with expert surgeons. One oft he goals of robotic telesurgical systems is to enable the surgeons to use open surgical techniques for suturing and knot tying in the MIS setting by having robotic tools with sufficient dexterity and a suitable user interface. Therefore, open surgical suturing motion data is used in the analysis. This way, it is possible to evaluate if the system can be used with the natural open surgical techniques, without the need of learning new ways to perform these tasks.

The goal of this project is the development of a robotic active catheter system for performing atrial fibrillation ablation under real-time inter-operative magnetic resonance imaging (MRI) guidance. The project synergistically integrates high-speed MRI technologies with robotic motion planning and control techniques to develop a novel co-robotic system. Specifically, the research focuses on i) development of new models and algorithms for robotic motion planning and control of active catheter systems, ii) development of algorithms for achieving real-time intra-operative MRI acquisition and image reconstruction, and, iii) hardware realization and experimental validation of the developed technologies.

For more information please visit MeRCIS Laboratory web pages, and our related publications.