Author | Kushner, Harold J. author |
---|---|
Title | Numerical Methods for Stochastic Control Problems in Continuous Time [electronic resource] / by Harold J. Kushner, Paul G. Dupuis |
Imprint | New York, NY : Springer US, 1992 |
Connect to | http://dx.doi.org/10.1007/978-1-4684-0441-8 |
Descript | X, 439 p. online resource |
1 Review of Continuous Time Models -- 1.1 Martingales and Martingale Inequalities -- 1.2 Stochastic Integration -- 1.3 Stochastic Differential Equations: Diffusions -- 1.4 Reflected Diffusions -- 1.5 Processes with Jumps -- 2 Controlled Markov Chains -- 2.1 Recursive Equations for the Cost -- 2.2 Optimal Stopping Problems -- 2.3 Discounted Cost -- 2.4 Control to a Target Set and Contraction Mappings -- 2.5 Finite Time Control Problems -- 3 Dynamic Programming Equations -- 3.1 Functionals of Uncontrolled Processes -- 3.2 The Optimal Stopping Problem -- 3.3 Control Until a Target Set Is Reached -- 3.4 A Discounted Problem with a Target Set and Reflection -- 3.5 Average Cost Per Unit Time -- 4 The Markov Chain Approximation Method: Introduction -- 4.1 The Markov Chain Approximation Method -- 4.2 Continuous Time Interpolation and Approximating Cost Function -- 4.3 A Continuous Time Markov Chain Interpolation -- 4.4 A Random Walk Approximation to the Wiener Process -- 4.5 A Deterministic Discounted Problem -- 4.6 Deterministic Relaxed Controls -- 5 Construction of the Approximating Markov Chain -- 5.1 Finite Difference Type Approximations: One Dimensional Examples -- 5.2 Numerical Simplifications and Alternatives for Example 4 -- 5.3 The General Finite Difference Method -- 5.4 A Direct Construction of the Approximating Markov Chain -- 5.5 Variable Grids -- 5.6 Jump Diffusion Processes -- 5.7 Approximations for Reflecting Boundaries -- 5.8 Dynamic Programming Equations -- 6 Computational Methods for Controlled Markov Chains -- 6.1 The Problem Formulation -- 6.2 Classical Iterative Methods: Approximation in Policy and Value Space -- 6.3 Error Bounds for Discounted Problems -- 6.4 Accelerated Jacobi and Gauss-Seidel Methods -- 6.5 Domain Decomposition and Implementation on Parallel Processors -- 6.6 A State Aggregation Method -- 6.7 Coarse Grid-Fine Grid Solutions -- 6.8 A Multigrid Method -- 6.9 Linear Programming Formulations and Constraints -- 7 The Ergodic Cost Problem: Formulations and Algorithms -- 7.1 The Control Problem for the Markov Chain: Formulation -- 7.2 A Jacobi Type Iteration -- 7.3 Approximation in Policy Space -- 7.4 Numerical Methods for the Solution of (3.4) -- 7.5 The Control Problem for the Approximating Markov Chain -- 7.6 The Continuous Parameter Markov Chain Interpolation -- 7.7 Computations for the Approximating Markov Chain -- 7.8 Boundary Costs and Controls -- 8 Heavy Traffic and Singular Control Problems: Examples and Markov Chain Approximations -- 8.1 Motivating Examples -- 8.2 The Heavy Traffic Problem: A Markov Chain Approximation -- 8.3 Singular Control: A Markov Chain Approximation -- 9 Weak Convergence and the Characterization of Processes -- 9.1 Weak Convergence -- 9.2 Criteria for Tightness in Dk [0, ?) -- 9.3 Characterization of Processes -- 9.4 An Example -- 9.5 Relaxed Controls -- 10 Convergence Proofs -- 10.1 Limit Theorems and Approximations of Relaxed Controls -- 10.2 Existence of an Optimal Control: Absorbing Boundary -- 10.3 Approximating the Optimal Control -- 10.4 The Approximating Markov Chain: Weak Convergence -- 10.5 Convergence of the Costs: Discounted Cost and Absorbing Boundary -- 10.6 The Optimal Stopping Problem -- 11 Convergence for Reflecting Boundaries, Singular Control and Ergodic Cost Problems -- 11.1 The Reflecting Boundary Problem -- 11.2 The Singular Control Problem -- 11.3 The Ergodic Cost Problem -- 12 Finite Time Problems and Nonlinear Filtering -- 12.1 The Explicit Approximation Method: An Example -- 12.2 The General Explicit Approximation Method -- 12.3 The Implicit Approximation Method: An Example -- 12.4 The General Implicit Approximation Method -- 12.5 The Optimal Control Problem: Approximations and Dynamic Programming Equations -- 12.6 Methods of Solution, Decomposition and Convergence -- 12.7 Nonlinear Filtering -- 13 Problems from the Calculus of Variations -- 13.1 Problems of Interest -- 13.2 Numerical Schemes and Convergence for the Finite Time Problem -- 13.3 Problems with a Controlled Stopping Time -- 13.4 Problems with a Discontinuous Running Cost -- 14 The Viscosity Solution Approach to Proving Convergence of Numerical Schemes -- 14.1 Definitions and Some Properties of Viscosity Solutions -- 14.2 Numerical Schemes -- 14.3 Proof of Convergence -- References -- List of Symbols