Projects | Tianyu Qiu

Inferring Occluded Agent Behavior in Dynamic Games from Noise Corrupted Observations

Sat, 13 Jul 2024 00:00:00 +0000

In mobile robotics and autonomous driving, it is natural to model agent interactions as the Nash equilibrium of a noncooperative, dynamic game. These methods inherently rely on observations from sensors such as lidars and cameras to identify agents participating in the game and, therefore, have difficulty when some agents are occluded. To address this limitation, this paper presents an occlusion-aware game-theoretic inference method to estimate the locations of potentially occluded agents, and simultaneously infer the intentions of both visible and occluded agents, which best accounts for the observations of visible agents. Additionally, we propose a receding horizon planning strategy based on an occlusion-aware contingency game designed to navigate in scenarios with potentially occluded agents. Monte Carlo simulations validate our approach, demonstrating that it accurately estimates the game model and trajectories for both visible and occluded agents using noisy observations of visible agents. Our planning pipeline significantly enhances navigation safety when compared to occlusion-ignorant baseline as well.

Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations

Thu, 01 Feb 2024 00:00:00 +0000

Many continuous control problems can be formulated as sparse-reward reinforcement learning tasks. In principle, online reinforcement learning methods can automatically explore the state space to solve each new task. However, discovering sequences of actions which lead to a non-zero reward becomes exponentially more difficult as the task horizon increases. Manually shaping rewards can accelerate learning for a fixed task, but it can be an arduous process that must be repeated for each new environment. This work introduces a systematic reward-shaping framework which distills the information contained in 1) a task-agnostic prior data set and 2) a small number of task-specific expert demonstrations, and then uses these priors to synthesize dense dynamics-aware rewards for the given task. This supervision substantially accelerates learning in our experiments, and we provide analysis demonstrating how the approach can effectively guide online learning agents to faraway goals.

Pedestrian Trajectory Prediction and Mobile Robot Navigation Based on Inverse Dynamic Games

Wed, 15 Mar 2023 00:00:00 +0000

With the development of relevant techniques in robotics and the maturity of application solutions, mobile robots are capable of working in more complicated scenarios and their application fields are enormously expanded. Service mobile robots tend to navigate in human-rich environments. Comparing with static obstacles, pedestrians are highly dynamic and tend to disturb the navigation of mobile robots. Navigating among pedestrians is thus challenging to mobile robots.

To tackle the inaccuracy of modelling pedestrian behaviors due to their high dynamic and randomness, we proposed an inverse dynamic game method based on model predictive control (MPC-GPred) to model pedestrians’ individual and group behavior.

For the high time complexity of solving for pedestrians’ trajectories caused by the nonlinear cost function, we did linear quadratic approximation and analytically solve for Nash strategy as pedestrian trajectory prediction. To achieve best prediction performance, we apply inverse dynamic game technique to identify weighting parameters according to trajectory observations. We tested our method with open-source algorithms RVO and CADRL on prediction performance and results indicate the effectiveness of our method.

In practical navigation tasks, to tackle the inaccuracy of modelling pedestrian behaviors due to interactions between pedestrians and robots, we proposed a dynamic game navigation method based on model predictive control (MPC-GNav) for mobile robots. we add the mobile robot as the new agent in dynamic games and solve for Nash strategy as the control for mobile robots. Consider the difference of dynamic model between mobile robots and the dynamic game model, we applied feedback linearization to achieve the control of robot platforms. To tackle the problem of incapability of control mobile robots in large scenarios, we proposed a layered planning method to obtain local goals. Mobile robots complete the entire navigation tasks by navigating to each local We compared our method with open-source algorithms RVO and CADRL in simulated scenarios and results indicate the effectiveness of our method in collision avoidance.

We implemented MPC-GNav on “Jiao Long” smart wheelchair in pedestrian-rich scenarios such as corridors and halls and did navigation experiments among pedestrians and compared the results with open-source method DWA. Results indicate the effectiveness of our method in improving navigation safeness.

Healthcare Telepresence Robot for the Elderly

Wed, 25 Dec 2019 00:00:00 +0000

This is our Capstone Design Project in 2019 Fall. We designed and built a healthcare telepresence robot for the elderly, which enables remote video chat and remote control of medicine dispensation.