Memory chatbot using AI Endpoints and LangChain4j

Have a look at our previous blog posts:
- Enhance your applications with AI Endpoints
- How to use AI Endpoints and LangChain4j
- LLMs streaming with AI Endpoints and LangChain4j
- How to use AI Endpoints and LangChain to create a chatbot
- How to use AI Endpoints, LangChain and Javascript to create a chatbot
- Create your own Audio Summarizer assistant with AI Endpoints!

The purpose of this blog post is to illustrate how to have a chatbot with memory. Indeed, with the previous example our chat can answer our questions one by one by forgetting the previous questions and answers.

: My name is Stéphane.
: Hello Stéphane, how can I assist you today?
: What is my name?
: I am not capable of knowing your personal information, including your name, 
as I don't have the ability to access such data. I'm designed to provide information based on the data 
I've been programmed with and my primary function is to assist with questions and provide relevant answers.

Not very useful in real life applications.

Fortunately, LangChain4j allows us to add memory to our chatbot.

In the following code we'll use the AI Services approach to simplify the code and use a memory object to store the chat history. In a real application, we should choose a more robust technology like a database for example.

We chose to use Mixtral-8x22b but feel free to use another LLM offered by AI Endpoints.

The first thing you need to do is add the dependencies in your pom.xml:

UTF-82121210.31.0dev.langchain4jlangchain4j${langchain4j.version}dev.langchain4jlangchain4j-mistral-ai${langchain4j.version}

Then, create your chatbot with memory as follows:

package com.ovhcloud.examples.aiendpoints;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import dev.langchain4j.memory.ChatMemory;
import dev.langchain4j.memory.chat.MessageWindowChatMemory;
import dev.langchain4j.model.mistralai.MistralAiStreamingChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.TokenStream;

public class MemoryStreamingChatbot {

  private static final Logger _LOG = LoggerFactory.getLogger(MemoryStreamingChatbot.class);
  private static final String OVHCLOUD_API_KEY = System.getenv("OVHCLOUD_API_KEY");

  interface Assistant {
    TokenStream chat(String message);
  }

  public static void main(String[] args) {
    MistralAiStreamingChatModel streamingChatModel = MistralAiStreamingChatModel.builder()
        .apiKey(OVHCLOUD_API_KEY)
        .modelName("Mixtral-8x22B-Instruct-v0.1")
        .baseUrl(
            "https://mixtral-8x22b-instruct-v01.endpoints.kepler.ai.cloud.ovh.net/api/openai_compat/v1/")
        .maxTokens(1500)
        .build();

    ChatMemory chatMemory = MessageWindowChatMemory.withMaxMessages(10);

    Assistant assistant = AiServices.builder(Assistant.class)
        .streamingChatLanguageModel(streamingChatModel)
        .chatMemory(chatMemory)
        .build();

    _LOG.info(": My name is Stéphane.\n");
    TokenStream tokenStream = assistant.chat("My name is Stéphane.");
    _LOG.info(": ");
    tokenStream
        .onNext(_LOG::info)
        .onComplete(token -> {
          _LOG.info("\n: What is my name?\n");
          _LOG.info(": ");
          assistant.chat("What is my name?")
              .onNext(_LOG::info)
              .onError(Throwable::printStackTrace).start();
        })
        .onError(Throwable::printStackTrace).start();



  }
}

⚠️ Don't forget to set en environment variable OVHCLOUD_API_KEY with your AI Endpoints token. ⚠️

You can test your new chatbot with its brand new memory:

: My name is Stéphane.
: Hello Stéphane, how can I assist you today?
: What is my name?
: Your name is Stéphane.

Don't hesitate to test our new product, AI Endpoints, and give us your feedback.

You have a dedicated Discord channel (#ai-endpoints) on our Discord server (https://discord.gg/ovhcloud), see you there!

ℹ️ You can find all the source code on out dedicated GitHub repository, public-cloud-examples.