Policy in Reinforcement Learning

In reinforcement learning, a policy is a function or strategy that determines the action an agent should take given a particular state. It essentially maps states to actions, guiding the agent’s behaviour in the environment. The policy can be deterministic or stochastic:

• Deterministic Policy: Directly maps each state to a specific action.

• Stochastic Policy: Maps each state to a probability distribution over actions.

The policy does not govern the environment. Instead, it determines how the agent interacts with the environment. The rules governing the environment are typically part of the environment’s dynamics or the transition model, not the policy.

The reward function is a separate component that defines how rewards are given based on the actions taken by the agent and the resulting state transitions. It evaluates the desirability of state-action pairs but is not the policy itself.

The initial state is where the agent starts, but it does not define how the agent decides on actions throughout the learning process. The policy determines the actions to be taken, not the initial state.