Making a case for LangChain

This Friday one of my lovely colleagues shared this article by Fabian Both from Octomind in our data science chat. The article joins a long list of articles, blogs and Reddit posts that dismiss LangChain. I'm personally partial to LangChain, but have tried to remain on the sidelines regarding this discussion. However, something snapped in me while reading the shared article and well... here we are with a big ranting blog post.

Let's try to dissect the core example that the article presents in its case against LangChain. First, a version using just the openai package is provided:

from openai import OpenAI


client = OpenAI()
text = "hello!"
language = "Italian"

messages = [
    {"role": "system", "content": "You are an expert translator"},
    {"role": "user", "content": f"Translate the following from English into {language}"},
    {"role": "user", "content": f"{text}"},
]

response = client.chat.completions.create(model="gpt-4o", messages=messages)
result = response.choices[0].message.content

Then, the LangChain version is presented¹:

from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate


model = ChatOpenAI(model="gpt-4o")
text = "hello!"
language = "Italian"

prompt_template = ChatPromptTemplate.from_messages(
    [("system", "You are an expert translator"),
     ("user", "Translate the following from English into {language}"),
     ("user", "{text}")]
)

parser = StrOutputParser()
chain = prompt_template | model | parser
result = chain.invoke({"language": language, "text": text})

The author claims that

All LangChain has achieved is increased the complexity of the code with no perceivable benefits.

While I agree with most of the statements in the article, I am in disagreement with the above statement and the rest of this blog will focus on showing some perceivable benefits.

To start off, let me agree that if your project's LLM usage is to make one call to OpenAI, then you should not use LangChain. My point will be that for more complex use cases, LangChain will show it's true value.

So let's add some complexity. While OpenAI provides a great service, they are occasionally down. Imagine our client asks us to implement a fallback to Anthropic in case OpenAI is down. What code changes would be necessary?

First, let's consider the case without LangChain:

from openai import OpenAI
+ from anthropic import Anthropic


client = OpenAI()
+ fallback_client = Anthropic()
text = "hello!"
language = "Italian"

messages = [
    {"role": "system", "content": "You are an expert translator"},
    {"role": "user", "content": f"Translate the following from English into {language}"},
    {"role": "user", "content": f"{text}"},
]

- response = client.chat.completions.create(model="gpt-4o", messages=messages)
- result = response.choices[0].message.content
+ try:
+     response = client.chat.completions.create(model="gpt-4o", messages=messages)
+     result = response.choices[0].message.content
+ except:
+     message = fallback_client.messages.create(
+         max_tokens=1024, model="claude-3-5-sonnet-20240620", messages=messages
+     )
+     result = message.content[-1]["text"]

Great, we managed to get it to work with minimal changes. Now how would this work with LangChain?

from langchain_openai import ChatOpenAI
+ from langchain_anthropic import ChatAnthropic
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

- model = ChatOpenAI(model="gpt-4o")
+ model = ChatOpenAI(model="gpt-4o").with_fallbacks(
+     [ChatAnthropic(model="claude-3-5-sonnet-20240620")]
+ )
text = "hello!"
language = "Italian"

prompt_template = ChatPromptTemplate.from_messages(
    [("system", "You are an expert translator"),
     ("user", "Translate the following from English into {language}"),
     ("user", "{text}")]
)

parser = StrOutputParser()
chain = prompt_template | model | parser
result = chain.invoke({"language": language, "text": text})

Note that we only needed to change the model definition and every other line remained unchanged.

Next, the client says we should detect the sentiment of the input text as well. We decide to use tool calling to get both the translation and sentiment with one LLM call.

Again, we start with Python. For simplicity's sake, let's forget about the Anthropic fallback for now:

from openai import OpenAI


client = OpenAI()
text = "hello!"
language = "Italian"

messages = [
    {"role": "system", "content": "You are an expert translator"},
    {
        "role": "user",
        "content": f"Translate the following from English into {language}",
    },
    {"role": "user", "content": f"{text}"},
]

- response = client.chat.completions.create(model="gpt-4o", messages=messages)
- result = response.choices[0].message.content
+ schema = {
+     "type": "function",
+     "function": {
+         "name": "Output",
+         "description": "",
+         "parameters": {
+             "type": "object",
+             "properties": {
+                 "translation": {"type": "string"},
+                 "sentiment": {
+                     "enum": ["positive", "negative", "neutral"],
+                     "type": "string",
+                 },
+             },
+             "required": ["translation", "sentiment"],
+         },
+     },
+ }
+ tool_choice = {
+     "type": "function",
+     "function": {"name": "Output"},
+ }
+ response = client.chat.completions.create(
+     model="gpt-3.5-turbo",
+     messages=messages,
+     tools=[schema],
+     tool_choice=tool_choice,
+ )
+ result = response.choices[0].message.tool_calls[0].function.arguments

Oof, are you still following the code? The result variable will now be a dict with translation and sentiment keys. Let's see what the changes would be for LangChain.

from langchain_openai import ChatOpenAI
+ from typing import Literal
+ from langchain_core.pydantic_v1 import BaseModel
- from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

+ class Output(BaseModel):
+     translation: str
+     sentiment: Literal["positive", "negative", "neutral"]

- model = ChatOpenAI(model="gpt-4o")
+ model = ChatOpenAI(model="gpt-4o").with_structured_output(Output)
text = "hello!"
language = "Italian"

prompt_template = ChatPromptTemplate.from_messages(
    [("system", "You are an expert translator"),
     ("user", "Translate the following from English into {language}"),
     ("user", "{text}")]
)

- parser = StrOutputParser()
- chain = prompt_template | model | parser
+ chain = prompt_template | model
result = chain.invoke({"language": language, "text": text})

Now the result variable will be an instance of the Pydantic Output class. LangChain's .with_structured_output abstraction with Pydantic is particularly concise and expressive in its intent.

Finally, the client tells us that we need to implement streaming, because they are tired of waiting without anything happening until the entire answer is generated. Also, they want this to work in conjunction with the Anthropic fallback and the simultaneous generation of the sentiment. Oh and your tech lead tells you that we need to use an observability framework like LangFuse to trace what is happening inside the chain.

I will leave the pure openai, anthropic and langfuse version of the code as an exercise for the reader. Here's the LangChain version:

from langchain_openai import ChatOpenAI
from langchain_anthropic import ChatAnthropic
from typing import Literal
from langchain_core.pydantic_v1 import BaseModel
from langchain_core.prompts import ChatPromptTemplate
from langfuse.callbacks import CallbackHandler

class Output(BaseModel):
    translation: str
    sentiment: Literal["positive", "negative", "neutral"]

callback = CallbackHandler()
model = (
    ChatOpenAI(model="gpt-4o").with_structured_output(Output)
).with_fallbacks(
    [ChatAnthropic(model="claude-3-5-sonnet-20240620").with_structured_output(Output)]
)
text = "hello!"
language = "Italian"

prompt_template = ChatPromptTemplate.from_messages(
    [
        ("system", "You are an expert translator"),
        ("user", "Translate the following from English into {language}"),
        ("user", "{text}"),
    ]
)

chain = prompt_template | model
for chunk in chain.stream(
    {"language": language, "text": text},
    config = {"callbacks": [callback]},
):
    print(chunk)

We could go on and try to tackle agents, multi-agents and whatever else we can dream of, but let's end the examples here. I hope the above examples show the benefits of using a framework like LangChain for developing LLM applications.

I'm interested to hear from you: do you still see "no perceivable benefits" of using LangChain? To me the answer is clear, LangChain's "expression language" and the abstractions it introduces allow for rapid development with code that clearly expresses its intent.

---

1. We made some small corrections to have valid code, such as defining the model variable for the LangChain code. Also we removed the API keys, which are assumed to be set as environment variables.

---