Revision as of 09:47, 29 May 2025

Brief of GenAI Chatbot

Introduction

GenAI Chatbot is a next-generation conversational AI assistant built on the OLLAMA architecture, supporting all models compatible with OLLAMA. Designed for seamless integration with GenAI Studio, it allows users to directly import models that have been fine-tuned within GenAI Studio, enabling easy deployment and immediate use of custom models in the chatbot. At its core, GenAI Chatbot utilizes efficient Small Language Models (SLMs) to provide natural, context-aware interactions. The chatbot features advanced capabilities, including audio processing (Speech-to-Text [STT], Text-to-Speech [TTS]), Retrieval-Augmented Generation (RAG), and an embedded vector database (VectorDB), all within a flexible configuration suitable for diverse application scenarios. It is currently optimized for embedded platforms such as NVIDIA Jetson Orin Nano and Jetson Orin AGX.

How To

Download SLM Models from GenAI Studio

1. Click GenAI Studio Hub from the left menu.
2. Enter the URL of your GenAI Studio.
3. Click the "Save" 
4. Once the configuration is successful, a notification will appear as shown at icon 4.
5. Displays a list of all models supported by GenAI Studio.
6. Click on icon 6 to download the desired model.

• After the download is complete, a notification will appear, and the icon will change to a completion icon.

After the model has downloaded, you can select this LLM Model in new Chat 

Create a new chatbot assistant with RAG

• Click on icon 1 to open the Workspace page.
• Click on icon 2 to open the Knowledge settings.
• Click on icon 3 to add new Knowledge.

• Enter the title at icon 1.
• Enter the content at icon 2.
• Click on icon 3 to create the Knowledge.

• The page will automatically switch back to the Knowledge section. After creation is complete, a notification will appear.

• Click on icon 1, the "+" icon.
• Click on icon 2, "Upload files."

• A file selection window will pop up. Select the files you want to use.

• After the upload, the files will be displayed and a success notification will appear.

• After the Knowledge is created, click on icon 1 to go to the Models page.
• Click on icon 2, the "+" icon, to add a new knowledge model.

• On the add new model page, fill in and select the required fields shown in the red box: Title, Subtitle, Base Model, and System Prompt.

• Continuing from the previous page, click on icon 1 to select the Knowledge you just created, then click on icon 2 to save and create the model.

• After successful creation, a notification will appear at icon 1. Then, click on the model at icon 2 to enter the model chat.

• The chat window will display that the model in use is the Knowledge model you created.

• After starting the conversation, you will see that the model retrieves information from the Knowledge you created in its responses.

Configuring TTS with Azure AI Speech API

Create an Account on Azure AI Speech API

1. Create an Azure account

• Get started with Azure’s free account: new users receive $200 credit for 30 days and free access to popular services.

• AI Speech – Text-to-Speech: 500,000 neural characters per month for free accounts

2. Set Up the Azure Speech Service

• In the Azure Portal, click on "Create a resource".

• Click on icon 1: "AI + Machine Learning"
• Then, click on icon 2: "Speech"

• Click on “Start” at the position marked by the red box.

• Click "Create" and fill in the necessary details:
  • Subscription: Choose your Azure subscription.
  • Resource Group: Select an existing group or create a new one.
  • Region: Choose a region close to your location.
  • Name: Provide a unique name for your Speech resource.
  • Pricing Tier: Select Free F0.
  • Click on icon 2: "Review + create"

• After confirming the information, click the “Create” button.

• After entering the overview page, click on the name link of the resource you created under "Resource."

• In the left-hand menu, click on "Keys and Endpoint".
• Note down the Key1 or Key2 and the Endpoint URL; you'll need these to authenticate your API requests.

Setting Up on GenAI Chatbot

• Select icon 1 and click on "Admin Panel."

• Find the TTS Settings section and select the Engine you want to use, as shown at icon 2: "Azure AI Speech"

• Enter the Azure AI Speech API token at icon 1.
• Click on icon 2, "Save."
• Finally, you will see a success notification at icon 3.

• Go to the chat window and click on icon 1, the conversation icon.
• An icon will appear at the position of icon 2, indicating that the feature is ready to use.

Evaluate the Benchmark of Each Chatbot Response

• An information button is provided next to each response. Clicking it reveals detailed performance and inference statistics for that response, including token counts, processing speed, and computation time. This allows developers to monitor and optimize system performance in real time.

Example

Creating an Audio + RAG Chatbot for Medication Assistant

1. Configuring TTS with Azure AI Speech API

2. Create Knowledge 

1. Client the Workspace from the left menu.
2. Go to the Knowledge
3. Click the + icon to add a new knowledge 

Here are sample files: * 'tial_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.pdf , * '10_Essential_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.txt  

 

 

1. Enter the title.
2. Enter the goal or description.
3. Click the "Create Knowledge" button to finish.

 

 

1. Click the "+" icon
2. Click the "Upload files" to upload files.  

 

After the upload, the files will be displayed and a success notification will appear.  

 

3. Create a Chatbot Assistant with RAG ( Knwledge )

1. Go to the Models tab.
2. Click the "+" icon, to add a new model.  

 

 

On the add new model page, fill in and select the required fields shown in the red box:

* Title, * Subtitle, * Base Model, and *System Prompt.  

 

 

Continuing from the previous page,

1. Click the "Select Knowledge" to select the Knowledge you just created,

2. Click the "Save & Create" to save and create.  

 

 

After successful creation, a notification will appear at icon 1.

Then, click on the model ( icon 2 ) to enter the model chat. Start using the audio assistant.

 

 

1. Make sure the select model is the RAG model.
2. Click the mic icon to start voice mode.
3. While speaking, the system will show a listening status.
4. After speaking, your question will appear as text.
5. The Medical Chatbot Assistant will response and play the answer with audio.