Stochastic Dynamic Programming

Course objective. This course provides a toolbox for solving dynamic optimization

problems in stochastic macroeconomic models. In particular, we briefly review optimal

control theory and dynamic programming. We then thoroughly study models in discrete

time and continuous time under uncertainty. The optimization problems are illustrated by

various examples of dynamic stochastic general equilibrium (DSGE) models.

Course outline.

Part I: Basic mathematical tools

(i) Control theory (maximum principle, Euler equation, transversality condition)

(ii) Dynamic programming (Bellman equation, envelope theorem, multiple variables)

(iii) An example: Lucas’ model of endogenous growth

Part II: Stochastic models in discrete time

(i) Stochastic control problems

(ii) Analyzing equilibrium dynamics

(iii) An example: Real business cycles (RBC)

(iv) An example: A new Keynesian model for monetary analysis

(v) Solving dynamic equilibrium models with Dynare

Part III: Stochastic models in continuous time

(i) Stochastic differential equations and rules for differentials (Itˆo’s formula)

(ii) An example: Merton’s model of growth under uncertainty

(iii) Stochastic dynamic control problems (Bellman equation)

(iv) An example: Continuous-time RBC (under Gaussian and/or Poisson uncertainty)

(v) An example: The matching approach to unemployment

(vi) An example: W¨alde’s model of endogenous growth cycles

Reading list. Sydsæter, Hammond, Seierstad, and Strøm (2008, chap. 4-12, 290 pages),

Chang (2004, chap. 4, 50 pages), Walde (2012), various articles suggested as complementary

material during the course

References

Chang, F.-R. (2004): Stochastic optimization in continuous time. Cambridge Univ. Press.

Sydsæter, K., P. Hammond, A. Seierstad, and A. Strøm (2008): Further Mathematics

for Economic Analysis. Prentice Hall.

Walde, K. (2012): Applied Intertemporal Optimization. Lecture Notes, Gutenberg University

Mainz, http://www.waelde.com/aio.