【作者】 朱方文；

【导师】 龚振邦； 韦穗；

【作者基本信息】 上海大学 ， 机械电子工程， 2005， 博士

【摘要】 本文讨论将计算机视觉用于人手姿态的检测,在基于任务建模的方式下,进行多指机械灵巧手的操作控制的研究工作。 机械手,包括灵巧手,是机器人进行操作的最重要的工具,对应的控制是机器人适应性地完成各种作业任务的关键。具有多手指、多关节和多自由度的多指灵巧机械手可以提高机器人的操作能力,实现复杂的灵巧操作。 传统上的机械手通常采用离线编程或示教的方法进行控制,机械手完成的是事先规定好的工作,这种工作模式降低了机械手对非结构环境的适应性。采用视觉传感器的机器人系统,用摄像机摄入被抓取的物体的影像,通过图像处理分析,获得物体的方位、形状参数,可以让机器人能自动识别需要抓取的对象,确定机械手的夹持器的方位和抓取方法。 但是,基于目前技术发展的水平,不可能完全靠传感器对机械手的直接控制来完成复杂的任务,即使在所谓结构环境中,机械手仍需要依赖多种形式的“学习”,“积累”处理可能出现的各种情形的知识。另一方面,在非结构化环境中,靠机器人完全适应性地完成各种作业任务是不现实的,往往需要人机配合,由人进行必要的监控和决策。并用人机交互手段干预机械手的动作。鼠标与键盘是传统上最常用的人机接口。 新型的人机接口装置——数据手套的出现,大大增加了机械手,特别是多指、多关节的机械灵巧手的适应性,但是数据手套多数是采用命令姿态作为控制方式的,操作者在使用数据手套以前需要进行训练,手套需要标定。命令姿态数量也是有限的。而且,数据手套是接触性传感器,容易损坏、不够舒适、价格比较高。另外,与计算机相连的数据手套在一定程度上限制了操作者的活动。手套开始工作后,就连续检测人手的动作,影响了操作者与其他人或设备的交互。 在过去的几年中,计算机视觉拓展了一个所谓“看人”(looking at people,以下简称LAP)的应用领域,用计算机视觉作为人机接口,通过观察(监视)、分析人体不同部分的表情或姿态,从中提取有用的控制信息。由于是非接触性检测,因此对于操作者而言感觉比较自由而且自然,同时还能为机器提供丰富的信息资源,具有广阔的应用范围。在Looking at people应用中,手势姿态分析主要研究内容是分析用手势表达的符号语言(如手语字符)的意义。 本文进行的研究将Looking at people的方法应用于分析人的手势姿态,进而完成多手指、多关节机械灵巧手的操作控制。这是一个交叉研究领域,涉及到计算机立体视觉、图像处理和机器人学等学科。 系统的特点是采用根据任务建模的方式而不是预先规定的命令姿态进行控制。由操作者观察被操作对象的表面形态、方位,作出动作决策(抓取物体的手势姿态),摄像机从不同角度“观察”人的手势,经过图像分析,信息合成,得到三维空间中手的姿态信息,然后利用人手和机械手结构上的异、同点,将人手的姿态转变成机械手可执行的动作姿态。 研究主要完成了下列工作: (1)根据人手的生理结构特点,设计了适合于具有小幅度运动的非刚性构架上特征点的搜索、排序方法,可以迅速、准确地将摄像机拍摄的处于任意空间方位姿态下的手的姿态图像上的各关节点(含手腕和指尖),按手指的归属重新排序,这些点在图像采集、更多还原

【Abstract】 Using free- hand gestures for remote control of objects an effective interaction way because the hand gestures are natural forms of communication and are easy to learn. A single gesture can be used to specify both a command and its parameters indicating the positions and movements of the hand and fingers, which provide a higher potential of expression. Using free hand as an input device eliminates the need for employing intermediate transducers.The ways for recognizing a hand gesture were classified as sensors-based and vision-based. Sensors-based ways use sensors physically attached to the hand to recognize the gesture, one of them is the "Data Glove". The mechanical data glove linked to the computer has the shortcoming of spoiling moving comfort and thus reduces the autonomy; in addition, the "glove" is expensive and could be damaged easily, the number of order gestures is also limited. Vision-based ways use optical sensors to detect the object, image processing and analyzing technologies were combined to perform the gesture recognition. The study about hand gesture analysis in vision-based system is mainly focused on the explanation of sign language. The research deals with steady positions recognition instead of dynamic gestures because a system that interprets gestures and translates them into a sequence of commands must have a way of segmenting the continuous stream of captured motion into discrete lexical entities, this process is somewhat artificial and necessarily approximate.The research introduced in this paper used "looking at people"(LAP), a new application area of computer vision in recent years, in the action control of a dexterous gripper with 3 fingers and 9 DOF. Instead of data glove, the system uses a pair of CCD camera and image processing and analyzing technologies for detecting the hand gesture of the operator.The camera pair observes the hand gesture of the operator from different directions and a pair of gesture images is caught. Image processing, image analyzing are performed to segment the key points on the hand and get the correlation of the points, stereo-vision theory is used to calculated the coordinates of the key points in the 3D space. The comparison between the physiological structure of the person hand and the mechanical structure of the dexterous gripper was performed to find a suitable way for transforming the information of the hand gesture to an executable parameter for the dexterous gripper.This is a subject-crossed research concerned with robotics, computer stereo vision, image processing and analyzing technologies. The main works done in the research are the following:A set of robust algorithms for detecting the key points positions on the hand and their co-relations was designed. It is suitable for the non-rigid structure with some small movement such as human’s hand. The algorithm could search all the key points on the hand under arbitrary orientation and rearrange them according to their host finger and their position on the fingers.On the basis of Lumigraph, the new theory in computer vision, an algorithm forcalculating the coordinates of the key points above in 3D space was developed. So the solid geometric calculations could be employed in the calculation of 3D information without calibrating the camera and the calculating the projection matrix, a complex matrix calculation process. Lining the space points up can restructure the hand gesture in 3D space.The research uses the task-based modeling in the control of the dexterous manipulator instead of command gesture. The operator observes the position and orientation of the object and design the grasping gesture for the mechanical gripper, shows it with his own hand. As introduced above, the system calculates the gesture information and the fingertip information of the operator was mapped as the fingertip position of the dexterous gripper. The inverse motion equation derived from the motion equation could be used to calculate the joints’ angles on 3 fingers.The vision-based grasping control of the dexterous gripper suggested in the paper provides the dexterous gripper with the direct grasping gesture instead of the order sequence in the data glove system, the fingertip mapping algorithm makes the gripper have precise fingertip position in the grasping action and can control the grasping for the objects with uncommon surface. Getting rid of a glove linked with computer makes the operator more free and comfortable. Any action of the hand beyond the monitoring area of the CCD will not affect the controlled device. The number of cameras in the system is not limited and the cameras in the system could be arbitrarily grouped as pairs without considering the relative position between them. This method was considered quite applicable for a multi-camera system, it could resolve the occlusion problem occurred in human-machine communication. This method is expected to be used in other research works, such as modeling a virtual hand in a virtual circumstance, tracking the motion of a human limb and fast 3D reconstruction of an object.更多还原