LangGraph: An Easy Introduction

INTRODUCTION

When developing an AI application, unorganized code might be fine for simple apps if they work correctly. However, as apps get more complex, managing disorganized code becomes much harder due to the many components and interactions involved. To tackle this, it's important to use a structured framework, like LangGraph, for building AI applications. LangGraph helps organize code efficiently, making it easier to add features and expand capabilities. This approach improves code readability, maintainability, and team collaboration, leading to more scalable and reliable AI applications.

LIBRARY & FRAMEWORK

TLDR - Libraries are tools, frameworks are structures.

In software development, a library is pre-written code for specific tasks, while a framework provides a structure for building applications. For example, 'Requests' is a Python library for API requests, and 'Django' is a Python framework for web apps. With a library, developers control the code flow; with a framework, the framework takes over it. Building an AI app with a library like Langchain can be disorganized and difficult to scale, so using a framework like Langgraph makes the code modular and easier to manage.

WHAT IS LANGGRAPH

LangGraph, created by the team behind LangChain, is an open source (Self hosted version is free but Cloud SaaS version is paid) AI agent framework designed to build, deploy and manage complex generative AI agent workflows. It provides a set of tools and libraries that enable users to create, run and optimize Large Language Models (LLMs) in a robust, scalable and efficient manner. At its core, LangGraph uses the power of graph-based architectures to model and manage the intricate relationships between various components of an AI agent workflow. Though it is built by the creators of LangChain, it can be used without LangChain library. LangGraph is heavily used in agentic AI. Tensorlake is an alternative to LangGraph (paid software and processes data in their own cloud but is highly scalable).

LANGGRAPH IN REAL WORLD

You can think of LangGraph as ‘n8n without a UI’. Here’s another trivia - the ‘graph’ in ‘LangGraph’ refers to the final structural output of the framework, where the application takes on the form of a graph data structure with nodes and edges. Therefore, in order to create a framework using LangGraph, we need to write code blocks (‘nodes’) that act as building blocks and edges that define the code flow. Since the code is divided into individual chunks, they can be run and tested independently. For state management within the framework, we declare a global state and maintain it from the very first node. For each graph invocation, in every node, we must provide a state before it starts and return the updated state once it finishes executing the code. LangGraph allows us to have a controlled autonomous flow with an orchestration framework for the AI, ensuring it stays within the loop you set up. It structures our code while offering flexibility.

QUICK DEMO

We are going to build an AI application which checks whether the user query is a coding related query or a simple one and route the users to appropriate models (i.e. OpenAI 4o for coding questions and any mini model for regular chat questions), after which it fetches the response and returns to the user. Let’s see how we can create this simple AI application using LangGraph (full code at the end of this aricle):

Install langgraph and other necessary libraries that we want in our framework like:
- TypedDict: for static type checking
- langgraph.graph: for creating langgraph framework
- typing: for type hints
- langsmith/langfuse: for tracing
- OpenAI library: to use their client to chat with LLM
- pydantic: for data validation (similar to TypeScript for JavaScript)
```
 from typing_extensions import TypedDict
 from langgraph.graph import StateGraph, START, END
 from typing import Literal
 from langsmith.wrappers import wrap_openai
 from openai import OpenAI
 from pydantic import BaseModel
```
Declare initial state of the graph
We can declare the initial state inside a ‘State’ class and setting the variables that we want to pass around and update via nodes
```
 class State(TypedDict):
     user_message: str
     ai_message: str
     is_coding_question: bool
```
Set the nodes & edges in the application
Our application should work like this:
- Start from the ‘detect_query’ node to detect the type of user query using ChatGPT
- Update variable ‘is_coding_question’ in the state based on detection
- Create a conditional edge which routes the application flow based on this variable in the state
- If the query is a coding query, route it to ‘solve_coding_question’ node and end the flow
- If the query is a regular query, route it to ‘solve_simple_question’ node and end the flow

Based on our analysis, we’d require the following nodes for our application:
- detect_query: to detect whether the user query is a coding query or not
- conditional_node: to route the application flow based on condition
- solve_coding_question: to solve the coding problems from the user
- solve_simple_question: to respond to regular user queries

Edges to be created are:
- START with ‘detect_query’ node
- detect_query’ to ‘conditional_node’
(‘conditional_node’ has the logic to route to ‘‘solve_coding_question’ or ‘solve_simple_question’)
- ‘solve_coding_question’ to END
- ‘solve_simple_question’ to END
It should look something like this:
Compile and run the graph!
Create a function which invokes the graph and then call this function.

Do note that ‘graph.invoke()’ function runs the code synchronously which blocks the code when one node is getting executed. We can use ‘graph.ainvoke()’ to run the code asynchronously.

That’s it. We just created our first framework built.

ADVANCED LANGGRAPH CONCEPTS

Advanced Langgraph concepts includes saving the state of a user interaction (checkpointing) and allow humans to provide input as part of the code logic (‘human in the loop’). As it is out of scope for this article, we will briefly touch upon them and cover them in-depth in another article.

THREADS

Simply put, a thread represents a user interacting with the AI application. It holds the current state (or data) during the execution of the AI application. A thread is a unique ID given to each checkpoint saved by a checkpointer. When using a checkpointer with the graph, you need to include a thread_id in the configurable section of the config like this:

{"configurable": {"thread_id": "1"}}

A thread's current and historical state can be retrieved. To persist state, a thread must be created prior to executing a run. The LangGraph Platform API provides several endpoints for creating and managing threads and thread state.

CHECKPOINTS

LangGraph has a built-in way to save user interaction data, using checkpointers. When you run a graph with a checkpointer, it saves a checkpoint of the graph's state at each step. These checkpoints are saved to a thread (as mentioned above), which you can access after the graph runs. Because threads let you see the graph's state after it runs, you can do things like involve humans, remember past states, go back in time, and handle errors. So essentially, a checkpoint is a snapshot of the graph's state saved at each step. Checkpoints are stored in a database (like MongoDB or PostgreSQL) and can be used to bring back the state of a thread later.

‘HUMAN-IN-THE-LOOP’

We can make use of the ‘human-in-the-loop’ features of LangGraph to allow human input at any point in a workflow, such as to review, edit, and approve tool calls in an agent or workflow. This is especially helpful in large language model (LLM)-driven applications where the model's output might need checking, fixing, or extra confirmation from an admin or the user themselves.

There are four typical design patterns that you can implement:

Approve or reject: Pause the graph before a crucial step, like an API call, to approve or reject the action.
Edit graph state: Pause the graph to review and edit the graph state, which is useful for correcting mistakes or updating the state with additional information, often involving human input.
Review tool calls: Pause the graph to review and edit tool calls requested by the LLM before tool execution.
Validate human input: Pause the graph to validate human input before proceeding with the next step.

Interrupts use LangGraph's persistence layer, which saves the graph state, to indefinitely pause graph execution until you resume. This is possible because LangGraph checkpoints the graph state after each step, which allows the system to persist execution context and later resume the workflow, continuing from where it left off. This supports asynchronous human review or input without time constraints. We can ‘interrupt’ to pause a graph from inside a node (dynamic), ‘interrupt_before’ and ‘interrupt_after’ to pause the graph at defined points (static) in the AI application execution.

CONCLUSION

In this article, we explored in detail how frameworks can significantly enhance the development of enterprise-level AI applications. We delved into the advantages of using structured frameworks, which provide a robust foundation for building scalable and efficient AI solutions. Specifically, we examined how LangGraph, a powerful tool in this domain, can be utilized to construct such applications. LangGraph offers a systematic approach to designing complex AI workflows, allowing developers to efficiently manage and execute various tasks. By leveraging its capabilities, we can streamline the development process, ensuring that AI applications are not only effective but also maintainable and adaptable to future needs. Through this discussion, we gained a comprehensive understanding of the pivotal role frameworks play in the realm of AI development. We have briefly touched upon some advanced concepts in Langgraph like checkpointing and ‘human-in-the-loop’ interruptions as well.

Here is a link to final code

REFERENCE

GenAI cohort by Hitesh Chaudhary and Piyush Garg

#10 - LangGraph: A Gentle Introduction

Table of contents