Robot Intelligence for Real World Applications

LIU Yunhui; ZHENG Fan; GUO Ruibin; WANG Jiangliu; NIE Qiang; WANG Xin; WANG Zerui

doi:10.1049/cje.2018.03.007

Volume 27 Issue 3

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Electronics > 2018 > 27(3): 446-458

LIU Yunhui, ZHENG Fan, GUO Ruibin, et al., “Robot Intelligence for Real World Applications,” Chinese Journal of Electronics, vol. 27, no. 3, pp. 446-458, 2018, doi: 10.1049/cje.2018.03.007

Citation:

LIU Yunhui, ZHENG Fan, GUO Ruibin, et al., “Robot Intelligence for Real World Applications,” Chinese Journal of Electronics, vol. 27, no. 3, pp. 446-458, 2018, doi: 10.1049/cje.2018.03.007

LIU Yunhui, ZHENG Fan, GUO Ruibin, et al., “Robot Intelligence for Real World Applications,” Chinese Journal of Electronics, vol. 27, no. 3, pp. 446-458, 2018, doi: 10.1049/cje.2018.03.007

Citation:

LIU Yunhui, ZHENG Fan, GUO Ruibin, et al., “Robot Intelligence for Real World Applications,” Chinese Journal of Electronics, vol. 27, no. 3, pp. 446-458, 2018, doi: 10.1049/cje.2018.03.007

PDF( 1932 KB)

Robot Intelligence for Real World Applications

doi: 10.1049/cje.2018.03.007

1.
T Stone Robotics Institute, The Chinese University of Hong Kong, China;
2.
National University of Defense Technology, Changsha 410073, China

Funds: This work is supported by the National Natural Science Foundation of China (No.U1613218), Hong Kong Research Grant Council (No.14204814), and Hong Kong Innovation and Technology Commission (No.ITS/112/15FP).

Received Date: 2018-01-02
Rev Recd Date: 2018-02-08
Publish Date: 2018-05-10

Abstract

Abstract

This paper presents a brief review on recent work on machine intelligence for real-world applications of robots. To act in a real world environment, a robot should possess a broad sense of intelligence including speech, perception, reasoning, action, etc. In this paper, we particularly deal with the intelligence involving action or body motion. The intelligence related to robot action/motion can be classified into two categories:manipulation intelligence and mobility intelligence. The manipulation intelligence means the skill/intelligence of reliably manipulating objects according to tasks and the mobility intelligence corresponds to the ability of autonomously moving, or flying, and or jumping in a natural environment. Human-robot interaction is another important topic for real-world applications. In addition to reviewing the major approaches, this paper also gives an overview on our efforts in these important topics.
- Machine intelligence,
- Simultaneous localization and mapping (SLAM),
- Medical robots,
- Humanrobot interaction

FullText(HTML)

References(103)

References

http://www.wikipedia.org/wiki/Artificial intelligence, 2018-3-23.

H. Gardner, Multiple Intelligences:Theory In Practice, Basic Books, 1993.

H. Durrant-Whyte and T. Bailey, "Simultaneous localization and mapping:Part I", IEEE Robotics and Automation Magazine, Vol.13, No.2, pp.99-110, 2006.

R. Kmmerle, et al., "G2o:A general framework for graph optimization", Proc. of IEEE International Conference on Robotics and Automation, Shanghai, pp.3607-3613, 2011.

S. Thrun, W. Burgard and D. Fox, "A probabilistic approach to concurrent mapping and localization for mobile robots", Autonomous Robots, Vol.5, No.3-4, pp.253-271, 1998.

M. Montemerlo, S. Thrun, D. Koller, et al., "FastSLAM:A factored solution to the simultaneous localization and mapping problem", Proc. of Eighteen National Conference on Artificial Intelligence, American Association for Artificial Intelligence, Menlo Park, CA, USA, pp.593-598, 2002.

P. Newman, J. Leonard, J.D. Tardos, et al., "Explore and return:experimental validation of real-time concurrent mapping and localization", Proc. of IEEE International Conference on Robotics and Automation, Washington, DC, USA, Vol.2, pp.1802-1809, 2002.

J.J. Leonard, R.J. Rikoski, P.M. Newman, et al., "Mapping partially observable features from multiple uncertain vantage points", The International Journal of Robotics Research, Vol.21, No.10-11, pp.943-975, 2002.

L. Carlone, et al., "A fast and accurate approximation for planar pose graph optimization", The International Journal of Robotics Research, Vol.33, No.7, pp.965-987, 2014.

W. Hess, D. Kohler, et al., "Real-time loop closure in 2D LIDAR SLAM", Proc. of IEEE International Conference on Robotics and Automation, Stockholm, Sweden, pp.1271-1278, 2016.

A.J. Davison, I.D. Reid, et al., "MonoSLAM:Real-time single camera SLAM", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.29, No.6, pp.1052-1067, 2007.

M.H. Li, B.R. Hong and R.H. Luo, "Mobile robot simultaneous localization and mapping using novel rao-blackwellised particle filter", Chinese Journal of Electronics, Vol.16, No.1, pp.34-39, 2007.

G. Klein and D. Murray, "Parallel tracking and mapping for small AR workspaces", 20076th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan, pp.1-10, 2007.

R. Mur-Artal, J.M.M. Montiel and J.D. Tards, "ORB-SLAM:A versatile and accurate monocular SLAM system", IEEE Transactions on Robotics, Vol.31, No.5, pp.1147-1163, 2015.

R.A. Newcombe, et al., "Dtam:Dense tracking and mapping in real-time", Proc. of IEEE Conference on Computer Vision, Barcelona, Spain, Vol.24, pp.2320-2327, 2011.

J. Engel, T. Schops and D. Cremers, "LSD-SLAM:Large-scale direct monocular SLAM", Proc. of European Conference on Computer Vision, pp.834-849, 2014.

J. Engel, V. Koltun and D. Cremers, "Direct sparse odometry", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.40, No.3, pp.611-625, 2018.

C. Forster, Z. Zhang, M. Gassner, et al., "SVO:Semidirect visual odometry for monocular and multicamera systems", IEEE Transactions on Robotics, Vol.33, No.2, pp.249-265, 2017.

R.F. Salas-Moreno, R.A. Newcombe, H. Strasdat, et al., "Slam++:Simultaneous localisation and mapping at the level of objects", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.1352-1359, 2013.

F. Liu, C. Shen, G. Lin, et al., "Learning depth from single monocular images using deep convolutional neural fields", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.38, No.10, pp.2024-2039, 2016.

Y. Hou, H. Zhang and S. Zhou, "BoCNF:Efficient image matching with bag of ConvNet features for scalable and robust visual place recognition", Autonomous Robots, No.9, pp.1-17, 2017.

R. Mur-Artal and J.D. Tards, "ORB-SLAM2:An open-source SLAM system for monocular, stereo, and RGB-D cameras", IEEE Transactions on Robotics, Vol.33, No.5, pp.1255-1262, 2017.

T. Whelan, et al., "Elastic fusion:Real-time dense SLAM and light source estimation", The International Journal of Robotics Research, Vol.35, No.14, pp.1697-1716, 2016.

A.I. Mourikis and S.I. Roumeliotis, "A multi-state constraint Kalman filter for vision-aided inertial navigation", Proc. of IEEE International Conference on Robotics and Automation, pp.3565-3572, 2007.

S. Leutenegger, et al., "Keyframe-based visualinertial odometry using nonlinear optimization", The International Journal of Robotics Research, Vol.34, No.3, pp.314-334, 2015.

R. Mur-Artal and J.D. Tards, "Visual-inertial monocular SLAM with map reuse", IEEE Robotics and Automation Letters, Vol.2, No.2, pp.796-803, 2017.

C. Forster, L. Carlone, F. Dellaert, et al., "On-manifold preintegration for real-time visual-inertial odometry", IEEE Transactions on Robotics, Vol.33, No.1, pp.1-21, 2017.

H. Wu, Y. Wu, C. Liu, et al., "Visual data driven approach for metric localization in substation", Chinese Journal of Electronics, Vol.24, No.4, pp.795-801, 2015.

A. Bicchi, "On the closure properties of robotic grasping", The International Journal of Robotics Research, Vol.14, No.4, pp.319-334, 1995.

M. Buss, H. Hashimoto and J. Moore, "Dexterous hand grasping force optimization", IEEE Transactions on Robotics and Automation, Vol.12, No.3, pp.406-418, 1996.

D. Ding, Y.H. Liu, et al., "An efficient algorithm for computing a 3D form-closure grasp", Proc. of IEEE/RSJ Conference on Intelligent Robotics and Systems, Vol.2, pp.1223-1228, 2000.

L. Han, J.C. Trinkle and Z. Li, "Grasp analysis as linear matrix inequality problems", IEEE Transactions on Robotics and Automation, Vol.16, No.6, pp.663-674, 2000.

M.L. Lam, D. Ding and Y.H. Liu, "Grasp planning with kinematic constraints", Proc. of IEEE/RSJ Conference on Intelligent Robotics and Systems, Vol.2, pp.943-948, 2001.

Y.H. Liu, "Qualitative test and force optimization of 3-D frictional form closure grasps using linear programming", IEEE Transactions on Robotics and Automation, Vol.15, No.1, pp.163-173, 1999.

Y.H. Liu, "Computing n-finger form-closure grasps on polygonal objects", The International Journal of Robotics Research, Vol.18, No.2, pp.149-158, 2000.

Y.H. Liu, et al., "A complete and efficient algorithm for searching for 3-D form-closure grasps in discrete domain", IEEE Transactions on Robotics, Vol.20, No.5, pp.805-816, 2004.

http://www.smarteyetech.com.

D. Henrich and H. Worn Eds., Robot Manipulation of Deformable Object (Series Advanced Manufacturing), SpringerVerlag, New York, USA, 2000.

V. Mallapragada, N. Sarkar and T. Podder, "Toward a robotassisted breast intervention system", IEEE/ASME Transactions on Mechatronics, Vol.16, No.6, pp.1011-1020, 2011.

J. Smolen and A. Patriciu, "Deformation planning for robotic soft tissue manipulation", Proc. of IEEE International Conference on Advances in Computer-Human Interactions, pp.199-204, 2009.

D. Sun and Y.H. Liu, "Modeling and impedance control of a two-manipulator system manipulating a flexible beam", ASME Journal of System, Dynamics, Measurement, and Control, Vol.119, pp.736-742, 1997.

D. Sun, J. Mills and Y.H. Liu, "Position control of multiple robots manipulating a general flexible object", The International Journal of Robotics Research, Vol.18, No.3, pp.319-332, 1999.

M. Higashimori, et al., "Active shaping of an unknown rheological object based on deformation decomposition into elasticity and plasticity", Proc. of IEEE International Conference on Robotics and Automation, pp.5120-5126, 2010.

M. Shibata and S. Hirai, "Soft object manipulation by simultaneous control of motion and deformation", Proc. of IEEE International Conference on Robotics and Automation, pp.2460-2465, 2006.

M. Shibata, "Wiping motion for deformable object handling", Proc. of IEEE International Conference on Robotics and Automation, pp.134-139, 2009.

Y.H. Liu and D. Sun, "Stabilizing a flexible beam handled by two manipulators via PD feedback", IEEE Transactions on Automatic Control, Vol.45, No.11, pp.2159-2164, 2000.

S. Hirai and T. Wada, "Indirect simultaneous positioning of deformable objects with multi-pinching fingers based on an uncertain model", Robotica, Vol.18, No.1, pp.3-11, 2000.

T. Wada, S. Hirai, S. Kawamura, et al., "Robust manipulation of deformable objects by a simple PID feedback", Proc. of IEEE International Conference on Robotics and Automation, Vol.1, pp.85-90, 2001.

J. Smolen and A. Patriciu, "Deformation planning for robotic soft tissue manipulation", Proc. of IEEE International Conference on Advances in Computer-Human Interactions, pp.199-204, 2009.

S. Tokumoto and S. Hirai, "Deformation control of rheological food dough using a forming process model", Proc. of IEEE International Conference on Robotics and Automation, Vol.2, pp.1457-1464, 2002.

J. Das and N. Sarkar, "Autonomous shape control of a deformable object by multiple manipulators", Journal of Intelligent and Robotic Systems, Vol.62, pp.3-27, 2011.

P. Zacharia, N. Aspragathos, I. Mariolis, et al., "A robotic system based on fuzzy visual servoing for handling flexible sheets lying on a table", Industrial Robot:An International Journal, Vol.36, No.5, pp.489-496, 2009.

G.L. Foresti and F.A. Pellegrino, "Automatic visual recognition of deformable objects for grasping and manipulation", IEEE Transactions on Systems, Man and Cybernetics, Vol.34, No.3, pp.325-333, 2004.

A.-M. Cretu, "Soft object deformation monitoring and learning for model-based robotic hand manipulation", IEEE Transactions on System, Man, and Cybernetics, Vol.42, No.3, pp.740-753, 2012.

H. Yoshida, T. Fukuda, M. Sakai, et al., "Manipulation of a flexible object by dual manipulators", Proc. of IEEE International Conference on Robotics and Automation, pp.318-323, 1995.

T. Yukawa, et al., "Handling of a constrained flexible object by a robot", Proc. of IEEE International Conference on Robotics and Automation, Vol.1, pp.324-329, 1995.

T. Kashiwase, M. Tabata, K. Tsuchiya, et al., "Shape control of flexible structures", Journal of Intelligent Material Systems and Structures, Vol.2, No.1, pp.110-125, 1991.

T. Yukawa, M. Uchiyama and D.N. Nenchev, "Stability of control system in handling of a flexible object by rigid arm robots", Proc. of IEEE International Conference on Robotics and Automation, Vol.3, pp.2332-2339, 1996.

M.J. Sullivan and N.P. Patpanikolopoulos, "Using active deformable models to track deformable objects in robotic visual servoing experiments", Proc. of IEEE International Conference on Robotics and Automation, Vol.4, pp.2929-2934, 1996.

F. Arai, R. Rong and T. Fukuda, "Trajectory control of flexible plate using neural network", Proc. of IEEE International Conference on Robotics and Automation, Vol.1, pp.155-160, 1993.

N. Sugita, F. Genma, Y. Nakajima, et al., " Adaptive controlled milling robot for orthopedic surgery", Proc. of IEEE International Conference on Robotics and Automation, Roma, Italy,pp.605-610, 2007.

A. Suebsomran and M. Pamichaknn, "Disturbance observerbased hybrid control of displacement and force in medical teleanalyzer for abdominal mass analysis", Proc. of IEEE International Conference on Industrial Technology, Vol.1, No.7, pp.365-369, 2002.

P. Valdastri1, S. Tognarelli1, A. Menciassi, et al., "A scalable platform for biomechanical studies of tissue cutting forces", Measurement Science and Technology, Vol.2673, No.10, pp.175-182, 2003.

C. Mendoza and C. Laugier, "Tissue cutting using finite elements and force feedback", Lecture Notes in Computer Science 2673, Surgery Simulation and Soft Tissue Modeling, pp.175-182, 2003.

S. Hutchinson, G.D. Hager and P.I. Corke, "A tutorial on visual servo control", IEEE Transactions on Robotics and Automation, Vol.12, No.5, pp.651-670, 1996.

A. Astolfi, L. Hsu, M. Netto, et al., "Two solutions to the adaptive visual servoing problem", IEEE Transactions on Robotics and Automation, Vol.18, No.3, pp.387-392, 2002.

Y.H. Liu, H. Wang, C. Wang, et al., "Uncalibrated visual servoing of robots using a depth-Independent interaction matrix", IEEE Transactions on Robotics, Vol.22, No.4, pp.804-817, 2006.

H. Wang, Y.H. Liu and D. Zhou, "Dynamic visual tracking for manipulators using an uncalibrated fixed camera", IEEE Transactions on Robotics, Vol.23, No.3, pp.610-617, 2007.

H. Wang, Y.H. Liu and D. Zhou, "Adaptive visual servoing using point and line features with an uncalibrated eye-in-hand camera", IEEE Transactions on Robotics, Vol.24, No.4, pp.843-857, 2008.

K. Wang, Y.H. Liu and L.Y. Li, "Visually servoed trajectory tracking of nonholonomic mobile robots without direct position measurement", IEEE Transactions on Robotics, Vol.30, No.4, pp.1026-1035, 2014.

K. Wang, Y. H. Liu and L. Li, "Vision-based tracking control of underactuated water surface robots without direct position measurement", IEEE Transactions on Control Systems Technology, Vol.23, No.6, pp.2391-2399, 2015.

D. Navarro-Alarcon, Y.H. Liu, J.G. Romero, et al., "Model-free visually servoed deformation control of elastic objects by robot manipulators", IEEE Transactions on Robotics, Vol.29, No.6, pp.1457-1468, 2013.

D. Navarro-Alarcon, Y.H. Liu, J.G. Romero, et al., "On the visual deformation servoing of compliant objects:uncalibrated control methods and experiments", The International Journal of Robotics Research, Vol.33, No.11, pp.1462-1480, 2014.

D. Navarro-Alarcon, H.M. Yip, Z. Wang, et al., "Automatic 3D manipulation of soft objects by robotic arms with adaptive deformation model", IEEE Transactions on Robotics, Vol.32, No.2, pp.429-441, 2016.

D. Navarro-Alarcon and Y.H. Liu, "Fourier-based shape servoing:A new feedback method to actively deform soft objects into desired 2D image contours", IEEE Transactions on Robotics, Vol.34, No.1, pp.272-279, 2017.

Z. Wang, S.C. Lee, F. Zhong,et al., "Image-based trajectory tracking control of 4-DOF laparoscopic instruments using a rotation distinguishing marker", IEEE Robotics and Automation Letters, Vol.2, No.3, pp.1586-1592, 2017.

D. Navarro-Alarcon, S. Singh, T. Zhang, et al., "Developing a compact robotic needle driver for MRI-guided breast biopsy in tight environments", IEEE Robotics and Automation Letters, Vol.2, No.3, pp.1648-1655, 2017.

H. Wang, A. Klser, C. Schmid, et al., "Action recognition by dense trajectories", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.3169-3176, 2011.

H. Wang and C. Schmid. "Action recognition with improved trajectories", Proc. of the IEEE International Conference on Computer Vision, pp.3551-3558, 2013.

J. Shotton, T. Sharp, A. Kipman, et al., "Real-time human pose recognition in parts from single depth images", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.1297-1304, 2011.

X. Yang and Y.L. Tian, "Effective 3d action recognition using eigenjoints", Journal of Visual Communication and Image Representation, Vol.25, No.1, pp.2-11, 2014.

C. Wang, Y. Wang and A.L. Yuille, "Mining 3D key-pose-motifs for action recognition", Proc. of the IEEE International conference on Computer Vision and Pattern Recognition, pp.2639-2647, 2016.

R. Vemulapalli, F. Arrate and R. Chellappa, "Human action recognition by representing 3d skeletons as points in a lie group", Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition, pp.588-595, 2014.

M.K. Pan, V. Skjervy, W.P. Chan, et al., "Automated detection of handovers using kinematic features", The International Journal of Robotics Research, Vol.36, No.1, pp.721-738, 2017.

K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition", arXiv preprint arXiv:1409.1556, 2014.

J. Wang, X. Nie, Y. Xia, et al., "Cross-view action modeling, learning and recognition", Proc. of the IEEE International Conference on Computer Vision and Pattern Recognition, pp.2649-2656, 2014.

S. Johnson and M. Everingham, "Clustered pose and nonlinear appearance models for human pose estimation", Proc. of the British Machine Vision Conference, pp.1-11, 2010.

M. Eichner, M. Marin-Jimenez, A. Zisserman, et al., "2D articulated human pose estimation and retrieval in (almost) unconstrained still images", International Journal of Computer Vision, Vol.99, No.2, pp.190-214, 2012.

B. Sapp, A. Toshev and B. Taskar, "Cascaded models for articulated pose estimation", Proc. of European Conference on Computer Vision, pp.406-420, 2010.

Y. Yang and D. Ramanan, "Articulated pose estimation with exible mixtures-of-parts", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.1385-1392, 2011.

D. Mehta, S. Sridhar, O. Sotnychenko, et al., "VNect:Realtime 3D human pose estimation with a single RGB Camera", arXiv preprint arXiv:1705.01583, 2017.

L.A. Schwarz, A. Mkhitaryan, D. Mateus, et al., "Human skeleton tracking from depth data using geodesic distances and optical flow", Image and Vision Computing, Vol.30, No.3, pp.217-226, 2012.

J. Tompson, A. Jain, Y. LeCun, et al., "Joint training of a convolutional network and a graphical model for human pose estimation", Proceedings of the International Conference on Neural Information Processing Systems, MIT Press, Vol.1, pp.1799-1807, 2014.

S.E. Wei, V. Ramakrishna, T. Kanade, et al., "Convolutional pose machines", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.4724-4732, 2016.

I. Akhter and M.J. Black, "Pose-conditioned joint angle limits for 3D human pose reconstruction", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.1446-1455, 2015.

R. Girshick, J. Donahue, T. Darrell, et al., "Rich feature hierarchies for accurate object detection and semantic segmentation", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.580-587, 2014.

R. Girshick, "Fast R-CNN", Proc. of IEEE International Conference on Computer Vision, pp.1440-1448, 2015.

S. Ren, K. He, R. Girshick, et al., "Faster R-CNN:Towards realtime object detection with region proposal networks", Advances in Neural Information Processing Systems, pp.91-99, 2015.

J. Redmon, S. Divvala, R. Girshick, et al., "You only look once:Unified, real-time object detection", Proc. of IEEE International Conference on Computer Vision and Pattern Recognition, pp.779-788, 2016.

W. Liu, D. Anguelov, D. Erhan, et al., "SSD:Single shot multibox detector", Proc. of European Conference on Computer Vision, pp.21-37, 2016.

R. Girshick, J. Donahue, T. Darrell, et al., "Region-based convolutional networks for accurate object detection and segmentation", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.38, No.1, pp.142-158, 2016.

S. Song and J. Xiao, "Sliding shapes for 3D object detection in depth images", Proc. of European Conference on Computer Vision, pp.634-651, 2014.

J. Redmon and A. Farhadi, "YOLO9000:Better, faster, stronger", arXiv preprint arXiv:1612.08242, 2016.

Relative Articles

Supplements(0)

Cited By

Proportional views