From handwritten to generated—Cohere's SDK journey

Cohere is one of the leading companies in today’s AI rush. Cohere is known for its Retrieval Augmented Generation (RAG) toolkit that allows LLMs to accurately answer questions and perform tasks. Developers use Cohere’s API to accomplish this—and, often, rely on its SDK to make those API calls.

Today, Cohere builds its SDKs on Fern. Prior to Fern, Cohere dealt with a classic SDK challenge. Like most dev-products, their users utilized a variety of languages. This created a design problem: each language had different type systems, primitives, and concurrency models. Accordingly, if the SDKs across languages were line-for-line alike, they would implicitly be compromising advantages boasted by each language.

For instance, TypeScript has a native type system. It supports convenient features like discriminated unions, helpful for handling deserialized objects in a type-safe manner. Other languages like Go have no such first-class language support. So Cohere needed to generate SDKs that offered symmetrical support for Cohere’s API with asymmetrical design patterns. Anything else would be short-sighted.

Fern enabled Cohere to deliver idiomatic SDKs without having to staff an SDK team. If we were to say it in CS 101 terms, Cohere reduced an O(n) problem to O(1) via Fern. Today, Cohere doesn’t have to spend time triaging hordes of small SDK errors. Instead, they focus on crafting a centralized, tightly-defined OpenAPI spec that encompasses requests, responses, errors, and examples. Cohere entrusts Fern to take care of the rest—SDK generation alongside publishing to targets like GitHub and package managers (e.g. npm, PyPI).

Let’s discuss Cohere’s implementation in detail.

Cohere used to rely on handwritten SDKs

Prior to discovering a robust automation tool like Fern, Cohere published a Go and Typescript SDK. However, both SDKs required significant maintenance, stealing (expensive) engineering time. In short, handwriting SDKs was simultaneously time-consuming and low-quality—human error creates inconsistent practices and poor code. Additionally, because SDKs were staffed by different engineers with different issue stacks, they were updated at different frequencies. This created a drift in feature coverage between SDKs.

This put Cohere in a tricky spot. SDKs had profuse errors and were collectively inconsistent. GitHub issues sometimes took months to resolve. Eventually, Cohere decided they needed to explore auto-generated SDKs. After evaluating severals tools, they settled on Fern.

Fern appealed to Cohere for a few reasons. First, it was open-source. This addressed some reasonable concerns about underlying transparency. It also dissolved fears of being commercially locked-in to a closed-source solution. Second, Fern worked with the widely accepted OpenAPI spec. And third—and most importantly—Fern could seamlessly generate well-designed, tightly-written SDKs for both Go and Typescript.

Within months, the results were obvious. GitHub issues were resolved in days. SDKs stayed auto-updated and consistent across repositories. This meant less errors, less developer hours, and better SDK experiences for users.

Today, Cohere continues to build its Typescript and Go SDKs via Fern. Let’s dive into how.

How Cohere uses Fern

Cohere manages an internal monorepo where they build their OpenAPI spec. OpenAPI, previously known as Swagger, is an open-source API specification. Cohere then syncs their OpenAPI repo with a separate repo centered around Fern’s specification file. From there, the SDKs are built, including the TypeScript SDK and Go SDK.

Cohere’s TypeScript SDK

The TypeScript SDK was Cohere’s first SDK re-vamp powered by Fern. Typescript was an apt choice given its wide-spread popularity for both frontend and backend applications.

Cohere’s TypeScript SDK can be installed via npm, yarn, or whatever package manager is preferred. It can be easily imported and configured with an API key thereafter.

import { CohereClient } from "cohere-ai";

const cohere = new CohereClient({
    token: "YOUR_API_KEY",
});

Cohere’s SDK offers async functionality. This is particularly helpful because AI processing often takes non-trivial time.

(async () => {
    const response = await cohere.chat({
        message: "Hello cohere!",
    });

    console.log("Response: ", response);
})();

Cohere’s Javascript SDK also has functionality to support streaming, directly plugging into streaming API endpoints.

(async () => {
    const stream = await cohere.chatStream({
        message: "Hello cohere!",
    });

    for await (const chat of stream) {
        if (chat.eventType === "text-generation") {
            process.stdout.write(chat.text);
        }
    }
})();

All of these functions are strongly-typed, making them easy to explore in modern text editors such as VSCode.

Cohere’s Go SDK

After TypeScript, Cohere launched Go. Go has more limitations than Typescript when it comes to type-safety. For instance, TypeScript supports discriminated unions—helpful for handling deserialized objects in a type-safe way. However, Go has no equivalent out-of-the-box support. Regardless, Fern makes it easy to export a Go SDK from the same origin as the TypeScript SDK’s, all without compromising TypeScript’s type-safety.

Like TypeScript, Go’s SDK is easy to import and configure.

import cohereclient "github.com/cohere-ai/cohere-go/v2/client"

client := cohereclient.NewClient(cohereclient.WithToken("<YOUR_AUTH_TOKEN>"))

After importing, it’s easy to access Cohere’s chat feature.

response, err := client.Chat(
  context.TODO(),
  &cohere.ChatRequest{
    Message: "How is the weather today?",
  },
)

Additionally, Cohere’s SDK makes it easy to use Go’s context primitive to set a timeout if the API doesn’t return a result fast enough.

ctx, cancel := context.WithTimeout(context.TODO(), time.Second)
defer cancel()

response, err := client.Chat(
  context.TODO(),
  &cohere.ChatRequest{
    Message: "How is the weather today?",
  },
)

Evidencing Fern’s consolidation benefits, Go’s streaming functions have perfect analogs to TypeScript’s.

stream, err := client.ChatStream(
  context.TODO(),
  &cohere.ChatStreamRequest{
    Message: "Please write a short story about the weather today.",
  },
)
if err != nil {
  return nil, err
}

// Make sure to close the stream when you're done reading.
// This is easily handled with defer.
defer stream.Close()

Even the aforementioned problem—Go’s diminished support for typed unions—is addressed by Fern. The Fern-generated SDK supports patterns that provide practical type safety.

For instance, the Cohere SDK makes each type a field on the event object. Therefore, developers can simply consume the event and check that the type of the sought-after struct is not nil , handling accordingly. It is, admittedly, a very verbose example, but this code snippet demonstrates exactly that:

package main

import (
    "context"
    "errors"
  "fmt"
    "io"

    cohere "github.com/cohere-ai/cohere-go/v2"
    cohereclient "github.com/cohere-ai/cohere-go/v2/client"
)

type StreamEventVisitor struct{
    ResponseTypes []string
}

func (v *StreamEventVisitor) VisitTextGeneration(e *cohere.ChatTextGenerationEvent) error {
    v.ResponseTypes = append(v.ResponseTypes, "text-generation")
    return nil
}

func (v *StreamEventVisitor) VisitStreamStart(e *cohere.ChatStreamStartEvent) error {
  v.ResponseTypes = append(v.ResponseTypes, "stream-start")
    return nil
}

func (v *StreamEventVisitor) VisitSearchQueriesGeneration(e *cohere.ChatSearchQueriesGenerationEvent) error {
  v.ResponseTypes = append(v.ResponseTypes, "search-queries-generation")
    return nil
}

func (v *StreamEventVisitor) VisitSearchResults(e *cohere.ChatSearchResultsEvent) error {
  v.ResponseTypes = append(v.ResponseTypes, "search-results")
    return nil
}

func (v *StreamEventVisitor) VisitCitationGeneration(e *cohere.ChatCitationGenerationEvent) error {
  v.ResponseTypes = append(v.ResponseTypes, "citation-generation")
    return nil
}

func (v *StreamEventVisitor) VisitStreamEnd(e *cohere.ChatStreamEndEvent) error {
  v.ResponseTypes = append(v.ResponseTypes, "stream-end")
    return nil
}

func main() {
    client := cohereclient.NewClient(cohereclient.WithToken("<YOUR_AUTH_TOKEN>"))

    stream, err := client.ChatStream(
        context.TODO(),
        &cohere.ChatStreamRequest{
            Message: "Please write a short story about the weather today.",
        },
    )

    if err != nil {
        panic(err)
    }

    // Make sure to close the stream when you're done reading.
    // This is easily handled with defer.
    defer stream.Close()

    // Construct a visitor to read the union response(s) from
  // the stream.
    var responseTypes []string
  visitor := &StreamEventVisitor{
        ResponseTypes: responseTypes,
    }

    for {
        message, err := stream.Recv()

        if errors.Is(err, io.EOF) {
            // An io.EOF error means the server is done sending messages
            // and should be treated as a success.
            break
        }

        // Do something with the message!
        message.Accept(visitor)
    }

  fmt.Printf("Received response types: %v\\n", responseTypes)
}

In short, Fern enables Cohere to publish a robust Go SDK without compromising on the quality of other SDKs.

Conclusion

Cohere’s switch to Fern demonstrated the benefits of automating SDK generation. Additionally, Fern’s robust feature-set enabled Cohere to maximize the quality of their SDKs for each respective language without compromising the cohesiveness of the SDK spec. And, because Fern is open-source and built-on OpenAPI, Fern posed no threats of vendor lock-in, making it a perfect choice for a quickly-growing organization like Cohere.

Whether it's TypeScript or Go, Fern has enabled Cohere to deliver high-quality, idiomatic SDKs. Today, Cohere’s users have better experiences utilizing their product.