LI Yanjie
Address:
D202O, HIT Campus, University Town of Shenzhen, Xili, Nanshan, Shenzhen, China (518055)
Email:
autolyj@hit.edu.cn
Phone:
PERSONAL PROFILE
 
RESEARCH INTEREST
 
Stochastic decision and optimization; Discrete event dynamic systems; Reinforcement learning; Unmanned vehicle control; Control and optimization of complex systems;
EDUCATION  
2001-2006
University of Science and Technology of China, Department of Automation, Master/PhD
1997-2001
Qingdao Univerity, Department of Mathematics, Bachelor
RESEARCH & WORK EXPERIENCE  
2010-Now
Harbin Institute of Technology, Shenzhen Graduate School, Associate Professor
2008-2010
Harbin Institute of Technology, Shenzhen Graduate School, Assitant Professor
2006-2008
Hongkong Univeristy of Science and Technology, Postdoctor
2013
University of New South Wales, Visiting Fellow
PROFESSIONAL QUALIFICATION & ACADEMIC SERVICE
 
2010-Now
IEEE Member
2012-Now
The Member of Operations Research Society of China
RESEARCH PROJECTS
2011-2013
National Natural Science Foundation: Sensitivity-based optimization of semi-Markov decision processes and its applications
2011-2012
Doctoral Fund of Ministry of Education: Performance sensitivity based hierarchical reinforcement learning
2011-2013
Shenzhen Basic Research: Intelligent storage system based on multi-robot
2011-2013
National Natural Science Foundation: Multi-robot simultaneous localization and mapping based on Robot Network
2012-2014
Shenzhen Basic Research: Transmission line detection for smart grid based on unmanned helicopter
RESEARCH ACHIEVEMENT & AWARDS
2014
Ho-Pan-Ching-Yi Best Paper Award for Discrete Even Dynamic Systems
2013
Shenzhen Peacock Plan
2006
Chinese Academy of Science Dean Reward
PATENT
   
PAPER & BOOK PUBLICATIONS
[1] Yanjie Li, Fang Cao,A basic formula for performance gradient estimation of Semi-Markov decision processes, European Journal of Operational Research, vol. 224, 333-339, 2013. (SCI)
[2] Yanjie Li, Baoqin Yin and Hongsheng Xi, Finding optimal memoryless policy of POMDPs under the expected average reward criterion, European Journal of Operational Research, vol. 211, 556-567, 2011. (SCI)
[3] Yanjie Li, Fang Cao and Xiren Cao, On-line policy gradient estimation with multi-step sampling, Discrete Event Dynamic Systems, vol. 20, 3-17, 2010. (SCI)
[4] Yanjie Li, Baoqun Yin and Hongsheng Xi, Partially observable Markov decision processes and performance sensitivity analysis, IEEE Transactions on System, Man and Cybernetics, Part B, vol. 38, no. 6, 1645-1651, 2008. (SCI)
[5] Baoqun Yin, Yanjie Li, Yaping Zhou and Hongsheng Xi, Performance optimization of semi Markov decision processes with discounted cost criteria, European Journal of Control, vol. 3, pp. 1-10, 2008. (SCI)
[6] Baoqun Yin, Guiping Dai, Yanjie Li, and Hongsheng Xi, Sensitivity analysis and estimates of the performance for M/G/1 queuing systems , Performance Evaluation, vol. 64, no. 4, pp. 347-356, 2007. (SCI)
[7] Guiping Dai, Baoqun Yin, Yanjie Li and Hongsheng Xi, Performance optimization algorithms based on potential for semi Markov control processes, International Journal of Control, vol. 78, no. 11, pp. 801-812, 2005. (SCI)
CONFERENCE PAPERS/TALKS
 
[1] Yanjie Li, An average reward performance potential estimation with geometric variance reduction, The 31th Chinese Control Conference, Hefei, 2012.
[2] Yanjie Li, Sensitivity-based optimization of semi-Markov decision processes, INFORMS International, Beijing, 2012.
[3] Yanjie Li, Reinforcement learning algorithms for semi-Markov decision processes, The 9th IEEE International Conference on Networking Sensing and Control, Beijing, 2012.
[4] Jianjun Li, Jiangong Ren and Yanjie Li, An Average-Reward Reinforcement Learning Algorithm based on Schweitzer’s Transformation, The 31st Chinese Control Conference,2012.
[5] Yanjie Li, Fang Cao, Inifnite horizon gradient estimation for semi-Markov decision processes, 8th Asian Control Conference, Kaohsiung, Taiwan, 2011.
[6] Wenwu Zeng, Xiaorui Zhu, Yanjie Li and Zexiang Li, Less Computational Unscented Kalman Filter for Practical State Estimation of Small Scale Unmanned Helicopters,IEEE International Conference on Robotics and Automation,2011.
[7] Yanjie Li and Fang Cao, An RVI reinforcement learning algorithm for semi-Markov decsion processes with average reward, World Congress on Intelligent Control and Automation, July, 2010.
[8] Qibo Liu, Yi Liu and Yanjie Li ,Combining sub-bands SNR on cochlear model for voice activity detection,International Conference on Asian Language Processing,2010.
[9] Hong Wang, Yanwen Xing, Yanjie Li and Zexiang Li,Dynamic modeling Of five-bar manipulator with structurally flexible linkages , World Congress on Intelligent Control and Automation ,2010.
[10] Yanjie Li, Fang Cao and Xiren Cao, An improvement of policy gradient estimation algorithm. In: Lennartson B, Fabian M, Akesson K, Guia A, Kumar R (eds), The Proceedings of Workshop on Discrete Event Systems, Goteborg, Sweden, pp. 168-172, 2008.
[11] Yanjie Li, Baoqun Yin and Hongsheng Xi, The policy gradient estimation of continuous-time hidden Markov decision processes, IEEE International Conference of Information Acquisition, Hong Kong, 2005.
TEACHING/SUPERVISING EXPERIENCE
Optimal Control (2012Spring, 2013Spring)
This course first introduces the related theory of convex optimization and its applications; then the theory and computation approaches are introduced, such as, calculus of variations, maximum principle and dynamic programming.

Reference Books:
[1] S. Boyd and L. Vandenberghe, Convex Optimization, NY: Cambridge University Press, 2004.
[2] A. E. Bryson and Y.-C. Ho, Applied Optimal Control, NY: Taylor & Francis, 1975.
[3] D.P. Bertsekas, Dynamic Programming and Optimal Control, Athena Scientific, 2007.
Convex Theory, Algorithms and Their Applications (2010Spring, 2011Spring)
This course introduces the related concepts of convex optimization, such as convex set, convex function and convex optimization. On these bases, the computation methods for convex optimization are introduced; Moreover, the applications of convex optimization are also introduced in the fields of control, image processing and communication.

Reference Book:
S. Boyd and L. Vandenberghe, Convex Optimization, NY: Cambridge University Press, 2004.
Linear Algebra (2009autumn, 2010autumn, 2011autumn)
This course is the upgrade of the corresponding undergraduate course. It introduces linear space, quotient space, duality, linear mapping, matices, determinant and trace and spectral theory.

Reference Book:
P. D. Lax, Linear Algebra and Its Applications, Second Edition, NJ: John Wiley & Sons, 2007.
Updated:2018-09