I’m wrapping up the UC Berkley’s LLM Agent MOOC course. For my writing assignment, I wanted to focus on the history of LLM agents. I am motivated to focus on LLM agent history because I was surprised during the week 2 lecture to learn about early systems that have existed since the 1960s, like the ELIZA chatbot. However, considering the definition of an agent, “intelligent” systems that interact with some “environment,” and how agents are a series of actions and observations, it makes sense that text agents existed in the mid-20th century. This post broadly covers the history of text agents from three phases: Symbolic AI agent, Reinforced Learning (RL) agent, and LLM agent, highlighting key advancements and persistent challenges.
Text agents have been around since the beginning of AI. Early text agents could interact with their environment and perform actions based on language input. ELIZA is a rule-based chatbot developed in the 1960s and marked an early milestone in the development of text agents. ELIZA is an example of the AI paradigm called symbolic AI agent, a paradigm spanning from 1960 through the 1990s. Like sophisticated reasoning engines, symbolic AI agents program all the rules to interact with different environments. ELIZA relied entirely on pre-programmed rules and used simple pattern-matching to create seemingly human-like conversational ability. ELIZA was limited by domain specificity and manual design and could only work up to simple domains. The lack of computational power during this era also prohibited the ability to do anything substantial to advance AI. While groundbreaking, these systems ultimately highlighted the need for more adaptable and intelligent approaches.
The emergence of machine learning and reinforcement learning (RL) ushered in the next phase for text agents starting from the mid-2000s to the present day. RL agents learn through interaction with their environments, receiving rewards for desirable actions and adjusting their behavior accordingly. This data-driven approach enabled agents to show greater flexibility and adaptability than their rule-based predecessors. However, pre-LLM RL agents were still limited by their domain specificity and reliance on clearly defined reward signals. Each new task often required extensive training from scratch, hindering their generalizability and scalability.
Powerful large language models (LLMs) exist today, and we now have reasoning agents that can understand complex instructions, plan actions, and adapt to circumstances. The advent of LLMs revolutionized the AI field, giving rise to LLM-powered reasoning agents. One example of an LLM agent is GPT-3, a machine learning model that uses deep learning to generate text that can seem like a human wrote it. These agents leverage the vast language understanding, reasoning, and knowledge inherent in LLMs to overcome the limitations of previous approaches.
Another LLM agent application I found exciting in the lecture is SWE-Bench, an LLM agent for code programming. In SWE-Bench (2023), presents a text agent with a GitHub repository and a specific issue. The agent needs to analyze the codebase, understand the problem, and generate a file diff that resolves the issue, just like a human software engineer would. This text agent goes beyond simply generating code snippets; it needs to interact with the code, run tests, and iterate on its solution.
Despite these advancements, LLM-powered reasoning agents still face challenges in developing intelligent text agents. One active research area is long-term memory, enabling agents to retain knowledge and learn from past experiences. Developing new benchmarks and evaluation metrics that go beyond simple task completion and emphasize robustness and human-like interaction is also crucial. As the field continues to evolve, the future holds exciting possibilities for text agents to become indispensable digital automation tools and problem-solving tools.
***Author's note: This article submission is to satisfy the written assignment for the Fall 2024 MOOC course, Learning Language Model Agents, facilitated by the UC Berkeley Center on Responsible Decentralized Intelligence (RDI) .***
