MA 359: Stochastic Dynamic Optimization
Credits: 3:0
Prerequisite courses: MA 361, MA 221, MA 231
The topic covered will be the control of discrete-time infinite state-space Markovian systems. These techniques appear frequently in the analysis and optimization of stochastic systems e.g. control of queues, resource allocation problems in networks, machine learning, reinforcement learning, operations research, etc. The course is aimed at students who work in applied probability, stochastic control, machine learning, networking. Course is divided into the following three parts:
- Control of Markovian systems that have countably infinite state-space.
- Continuous State-Space Systems: Measurability Questions, Control under Monotonicity Assumption, Control under Contraction Assumption. Borel Models, Borel Spaces, Analytic Sets, Imperfect State Observations Model.
- General (Irreducible) Markov Chains: Kernels, Transience and Recurrence, Embedded Renewal Processes, Positive and Null Recurrence, Control of Harris Chains.