“Sliding Markov Decision Processes for Dynamic Task Planning on Uncrewed Aerial Vehicles”

Mission and flight planning problems for uncrewed aircraft systems (UASs) are typically large and complex in space and computational requirements. With enough time and computing resources, some of these problems may be solvable offline and then executed during flight. In dynamic or uncertain environments, however, the mission may require online adaptation and replanning. In this work, we will discuss methods of creating MDPs for online applications, and a method of using a sliding resolution and receding horizon approach to build and solve Markov Decision Processes (MDPs) in practical planing applications for UASs. In this strategy, called a Sliding Markov Decision Processes (SMDP), the underlying state space is regularly rediscretized according to its informational proximity and utility while a receding horizon algorithm allows us to consider immediate next steps while keeping the primary goal state in mind. This approach allows for dynamic decision making and replanning by a UAS in an uncertain and dynamic environment in which mission objectives or the environment could change. The SMDP method shows an ability to create recursively optimal policies, under conditions of limited computing power and time, that perform similarly to the optimal policy of the associated fully-modeled flat MDP.

Committee:
Dr. Justin Bradley, Advisor
Dr. Carl Nelson
Dr. Bhuvana Gopal