1. Learn
    2. /
  2. Courses
    3. /
  3. Reinforcement Learning with Gymnasium in Python
Connected

Exercise

Implementing value iteration

Value iteration is a key method in RL for finding the optimal policy. It iteratively improves the value function for each state until it converges, resulting in the discovery of the optimal policy. You'll start with an initialized value function V and policy, both preloaded for you. Then, you'll update them in a loop until the value function converges and see the policy in action.

The get_max_action_and_value(state, V) function has been pre-loaded for you.

Instructions

100 XP
  • For each state, find the action with the maximum Q-value (max_action) and its corresponding value (max_q_value).
  • Update the new_V dictionary and the policy based on max_action and max_q_value.
  • Check for convergence by checking if the difference between new_v and V for every state is less than threshold.