Condition state-based decision making in evolving systems: Applications in asset management and delivery

Decision making in stochastic dynamic systems is significantly different from decision making in deterministic systems in that agents need to make multiple management or operational decisions along a time horizon, and each decision considers not only current influences, but also temporal impacts on the system. The evolving nature of conditions in a dynamic system over time and uncertainty in condition transition and observation increase the difficulties associated with making good decisions in these environments. This dissertation investigates aspects of decision making for evolving systems with two applications in the transportation domain: roadway asset management and meal delivery. To this end, it: proposes a bilevel, stochastic, dynamic program with embedded Markov decision process (MDP) and traffic user equilibrium, along with an actor-critic-based deep reinforcement learning (DRL) solution method, for prioritizing and scheduling potential roadway improvement actions across asset classes given evolving condition state;models a maintenance and rehabilitation (M&R) scheduling problem given only partially and imperfectly observed conditions and nonstationary condition deterioration probabilities as a partially observable MDP and, through a DRL solution methodology, investigates the potential gains from scheduling roadway M&R actions in response to continuously updated, low-quality sensor- and intermittent, high-precision, inspection-based condition-state information;develops a chance-constrained, bilevel mathematical model that with condition value at risk (CVaR) approximation determines an optimal roadway maintenance and resurfacing scheduling that ensures an acceptable level of reliability for traffic network users;builds on a stochastic, discrete-event simulation (DES) platform with tabu search heuristic and embedded ejection chain approach for optimal meal delivery job bundling, routing and assignment over a rolling horizon to replicate the dynamics of the meal delivery setting and, through a CVaR measure, evaluate risk of late delivery due to the use of an at-will workforce; andconceptualizes the problem of determining whether an at-will driver working in a meal delivery environment should accept an order offer or wait for a better offer, and whether and where to move when idle as an MDP, and tests two DRL methodologies to determine the courier’s best decisions to take as the courier’s shift progresses.These contributions build on cutting-edge methods of DRL, stochastic optimization and stochastic simulation to push the boundaries of current knowledge in decision-making in dynamic and stochastic systems toward enhanced performance, efficiency and reliability. The dissertation provides the mathematical and algorithmic underpinnings to support decision-making in real-world, complex environments, where condition states evolve stochastically and making good decisions is very difficult.