1. Learn
    2. /
  2. Courses
    3. /
  3. Deep Reinforcement Learning in Python
Connected

Exercise

Critic network

Actor Critic methods require two very different neural networks.

The architecture for the actor network is identical to that of the policy network you used for REINFORCE, so you can reuse the PolicyNetwork class.

However, the critic network is something you haven't implemented so far. The critic aims to approximate the state value function \(V(s_t)\), rather than the action value function \(Q(s_t, a_t)\) approximated by Q-Networks.

You will now implement the Critic network module which you will use in A2C.

Instructions

100 XP
  • Fill in the desired dimension for the second fully connected layer so that it outputs one state value.
  • Obtain the value returned by the forward pass through the critic network.