高级宏观经济学讲义英文版朱晓东

Lecture Notes: Advanced MacroeconomicTheoryXiaodong ZhuJanuary, 20051Introduction to Dynamic ProgrammingIn this section we provide a heuristic introduction to the ideas of dynamicprogramming. (By heuristic we mean that we will provide no proofs andnot worry about the regularity conditions that are needed for some of thestatements we make.) Consider the optimization problem in the neoclassicalgrowth model:max fct;kt+1gt?01Xt=0?tU(ct)subject toct? 0; kt+1? 0,t = 0;1;:ct+ kt+1= f(kt),t = 0;1;:Let ?(k) = 0;f(k). We can rewrite the optimization problem asv(k0) =max kt+12?(kt),t=0;1;:1Xt=0?tU(f(kt) ? kt+1)(P)The question here is how to solve the innite-dimention optimizationproblem above. To get ideas, let us rst look at a corresponding problemwith nite number of periods:v(T)(k0) =max kt+12?(kt),t=0;:;TTXt=0?tU(f(kt) ? kt+1)(P0)Note that the objective function in (P0) can be written asTXt=0?tU(f(kt) ? kt+1) = U(f(k0) ? k1) + ?TXt=1?t?1U(f(kt) ? kt+1):1and (P0) can be written as1v(T)(k0) =max k12?(k0)“U(f(k0) ? k1) + ?max kt+12?(kt),t=1;:;TTXt=1?t?1U(f(kt) ? kt+1)#:Note that the maximization problem in the bracket is the optimization prob-lem in period 1 when k1is given. Lettingv(T?1)(k1) =max kt+12?(kt),t=1;:;TTXt=1?t?1U(f(kt) ? kt+1);then, we have the following recursive equationv(T)(k0) =max k12?(k0)?U(f(k 0) ? k1) + ?v(T?1)(k1)?:(R0)More generally, if we denev(T?n)(k) =max kt+12?(kt),t=n;:;T;kn=kTXt=n?t?nU(f(kt) ? kt+1):(Here, the sub-index T ? n refers to the number of periods remaining afterthe current period.) Then, we havev(T?n)(k) = max k02?(k)?U(f(k) ? k0) + ?v (T?n?1)(k0)?;n = 0;1;:;T ? 1:(R)1Here, we used the following mathematical fact: For a real function F(x;y), where xand y are vectors, letting A be a constraint set for x and B(x) be a constraint set for y given any x 2 B, then,max x2A;y2B(x)F(x;y) = max x2A maxy2B(x)F(x;y):In the problem (P0), consider x = k1and y = (k2;:;kT+1).2There is no future value to worry about in the terminal period T, so theterminal value function is simply:v(0)(k) = max k02?(k)U(f(k) ? k0):(PT)Using (PT) and the equation (R) we can solve for v(T)recursively.So, the idea of dynamic programing is to turn a multi-period dynamicoptimization problem into a series of simple one-period optimization problem.This method works well for a problem with nite number of periods since thelast period s value function is always well specied. What about problemswith innite number of periods? In that case, there would be no nal period.Where should we start in the recursive process?The innite period optimization problem in (P) can be thought of as thelimit of the nite period problems when T approaches innity, that isv(k) = lim T?!1v(T)(k) = lim T?!1v(T?1)(k):Applying the limit on both sides of (R0) we then have2v(k) = max k02?(k)U(f(k) ? k0) + ?v(k0):(B)In other words, the value function of the innite-period problem is a solutionto a functional equation given in (B), which is often called the BellmanEquation. One of the main subjects that we will discuss in dynamic pro-2Note that here we have changed k0to k and k1to k0, but these are just notational changes.3gramming is how to solve such a functional equation. There are generallythree di¤erent approaches:? (i) First order conditions. Use some methods to show that the Bell-man Equation has a (unique) solution that satises certain conditionssuch as continuity, di¤erentiability, etc. Then, use rst order conditionsand envelope theorem to analyze the properties of the optimal solution.? (ii) Value function iteration. Solve the Bellman Equation numeri-cally. In this case, one can often start from an arbitrary initial functionv0and iterate according to the following equationvn(k) = max k02?(k)fU(f(k) ? k0) + ?vn?1(k0)guntil the sequence of functions fvngn?0converges to a limiting function.It can often be shown that the limiting function is the solution tothe Bellman equation (B). Note that the sub-index n here does notrepresent the number of remaining periods in the optimization problem.Rather, it refers to the value function of nth iteration for the sameinnite horizon problem.? (iii) Guess and verify. Guess a functional form for v (x) and showthat it is indeed the solution to the Bellman Equation. In this case,everything can be solved analytically.We now show how each of these three methods can be applied to theneoclassical growth model.4Example 1.(First order condition method).First, assume that theBellman Equation has a solution that is di¤erentiable, and that the optimalk02 (0;f(k)3. Then, from the rst-order condition of the maximizationproblem in the Bellman Equation, we haveU0(f(k) ? k0) = ?v0(k0):If we know the expression for v0, then, the above equation can be used tosolve for the optimal capital stock k0. Let k0= h(k) be the optimal solution,then, by denition,v(k) = U(f(k) ? h(k) + ?v(h(k)which implies thatv0(k) = U0(f(k) ? h(k)f0(k) ? U0(f(k) ? h(k)h0(k) + ?v0(h(k)h0(k):However, from the r