Learning Agent

An improvement over basic agents. Learns from its experiences to improve future behavior.

Components

Performance element

Takes in percepts and decides on actions. Corresponds to what was called “the agent” in prior models.

Critic

Evaluates how the agent is performing against the performance standard and produces feedback signals.

Learning element

Takes feedback from the critic and modifies the performance element to improve future decisions.

Problem generator

Suggests exploratory actions that lead to new and informative experiences. These actions may be suboptimal in the short term but yield better overall knowledge.

Component Interaction

Percepts → Performance Element → Actions
               ↑                    ↓
         Learning Element      Environment
               ↑                    ↓
            Critic ← ← ← ← ← ← Feedback
               ↑
        Problem Generator

The critic judges action quality against the performance measure. The learning element updates the performance element accordingly. The problem generator injects exploration to avoid local optima.

Exploration vs. Exploitation

• Exploitation
  Use current knowledge to take the best-known action. Maximizes short-term performance.
• Exploration
  Try new actions to gather information. May reduce short-term performance but improves long-term knowledge.

The problem generator drives exploration. Balancing the 2 is a central challenge in reinforcement learning.