AI4OS LLM

As part of the tool repertoire that the AI4OS stack is offering to increase our user’s productivity, we are current offering an LLM Chatbot service (in beta!) that allows users to summarize information, get code recommendations, ask questions about the documentation, etc.

We care about user privacy, so it’s important to notice that your chat history will be erased whenever you delete it, and no data will be retained by the platform (privacy policy).

Login

The service is available at: https://llm.dev.ai4eosc.eu

To access the LLM service, you first need to register in the platform.

../_images/login.png

Once you login, you will arrive to a landing page where you will be able to select the model which you want to interact with.

../_images/landing.png

The current available models are:

  • Nemo-2407: a small-size model from the Mistral family (12B parameter) with a large context window (128K tokens), released on July 2024,

  • Small-2409: a medium-size model from the Mistral family (22B parameters) with a smaller context window (32K tokens), released on September 2024,

  • Assistant: our custom model designed to help you navigate our documentation,

Now, let’s explore some common usages of the tool. Keep in mind that the AI4OS LLM is built with OpenWebUI so you always find further information in their documentation.

Chat with the LLM

We can ask generic questions to the model.

../_images/chat.png

Remember that if you answer relies on up-to-date information, you can always enable Web search under the button.

Summarize a document

Under the button, you can select Upload files. This will allow you to query a document with questions.

../_images/upload-files.png

Ask questions about the documentation

Important

This service is currently under development, so it might not be accessible to you.

In the upper left corner, you can select the AI4EOSC/Assistant model to ask questions about the platform. The LLM with use our documentation as knowledge base to provide truthful answers to your questions.

../_images/assistant.png

Integrate it with your own services

Under Settings Account you will be able to generate your own API keys to access the service. Learn more on how to use API keys to integrate the AI4OS LLM into your own services (endpoints are compatible with the OpenAI API spec).

../_images/api-keys.png