Latest revision as of 03:42, 26 September 2025

Introduction

GenAI Chatbot is a next-generation conversational AI assistant designed to provide natural, context-aware interactions. At its core, it utilizes efficient Small Language Models (SLMs) and supports the direct import of models fine-tuned within GenAI Studio, enabling easy deployment and immediate use of custom models in the chatbot.

The chatbot features advanced capabilities, including:

Audio processing (Speech-to-Text [STT] and Text-to-Speech [TTS]).
Retrieval-Augmented Generation (RAG).
An embedded vector database (VectorDB).
Model Context Protocol (MCP) tools, which enable AI agents to perform actions.

GenAI Chatbot features a flexible configuration suitable for diverse application scenarios and is optimized for embedded platforms such as the Ryzen AI 8000, NVIDIA Jetson Orin Nano, and Jetson Orin AGX.

How To

Download Models from Ollama (Support Device: Nvidia)

Go to the page https://www.ollama.com/search and click on the "Models" tab as shown in Icon 1.
Use the search bar shown in Icon 2 to find the model you want to download.
In Icon 3, locate the name of the model.

After clicking in, you'll see the name and version of the model you need to download.
Please note that the model size depends on your hardware resources.
Make sure your hardware matches the specifications shown in the table above.

Go back to GenAI-Chatbot and create new chat window, as shown in Icon 1.
Enter the model name you just saw into the search bar in the chat window, as shown in Icon 2.
Then, click the download button at the location marked in Icon 3.

After clicking the download button, a notification will pop up.

After the download is complete, a notification will appear confirming the completion.

Next, you'll be able to find the model you downloaded in the model selection menu.

Delete Models on GenAI-Chatbot (Support Device: Nvidia)

⚠️ Delete Models For AMD

Click on icon 1 "User" to enter the Admin Panel (icon 2).
After entering "Settings," click on icon 3 "Models."
Click the download icon (icon 4) on the right.

After the window pops up, find the option to delete the model at the location of icon 1.
After selecting the model you want to delete, click on icon 2 "Delete."

Click "Confirm".

A notification will pop up after a successful operation.

Download SLM Models from GenAI Studio

Click GenAI Studio Hub from the left menu.
Enter the URL of your GenAI Studio.
Click the "Save"
Once the configuration is successful, a notification will appear as shown at icon 4.
Displays a list of all models supported by GenAI Studio.
Click on icon 6 to download the desired model.

After the download is complete, a notification will appear, and the icon will change to a completion icon.

After the model has downloaded, you can select this LLM Model in new Chat

Create a new Knowledge

Client the Workspace from the left menu.
Go to the Knowledge.
Click the + icon to add a new knowledge.

Here are sample files: * PDF: tial_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.pdf , * Text: 10_Essential_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.txt

Enter the title.
Enter the goal or description.
Click the "Create Knowledge" button to finish.

Click the "+" icon.
Click the "Upload files" to upload files.

Select the files you want to use.

After the upload, the files will be displayed and a success notification will appear.

Create a Chatbot Assistant with RAG ( Knowledge )

Go to the Models tab.
Click the "+" icon, to add a new model.

On the add new model page, fill in and select the required fields shown in the red box:
- Click the "+" icon, to add a new model.
- * Title, * Subtitle, * Base Model, and * System Prompt.

Continuing from the previous page,

Click the "Select Knowledge" to select the Knowledge you just created,
Click the "Save & Create" to save and create.

After successful creation, a notification will appear at icon 1.
Then, click on the model ( icon 2 ) to enter the model chat. Start the Assistant chat.

The chat window will display that the model in use is the Knowledge model you created.

After starting the conversation, you will see that the model retrieves information from the Knowledge you created in its responses.

Configuring TTS with Azure AI Speech API

Create an Account on Azure AI Speech API

1. Create an Azure account

Get started with Azure’s free account: new users receive $200 credit for 30 days and free access to popular services.

AI Speech – Text-to-Speech: 500,000 neural characters per month for free accounts

2. Set Up the Azure Speech Service

In the Azure Portal, click on "Create a resource".

Click on icon 1: "AI + Machine Learning"
Then, click on icon 2: "Speech"

Click on “Start” at the position marked by the red box.

Click "Create" and fill in the necessary details:
- Subscription: Choose your Azure subscription.
- Resource Group: Select an existing group or create a new one.
- Region: Choose a region close to your location.
- Name: Provide a unique name for your Speech resource.
- Pricing Tier: Select Free F0.
- Click on icon 2: "Review + create"

After confirming the information, click the “Create” button.

After entering the overview page, click on the name link of the resource you created under "Resource."

In the left-hand menu, click on "Keys and Endpoint".
Note down the Key1 or Key2 and the Endpoint URL; you'll need these to authenticate your API requests.

Setup the Azue Text-to-Speec in GenAI Chatbot

Click the"Admin Panel."

Click the Audio
Select the "Text-to-Speech Engine" with "Azure AI Speech".

Enter the Azure AI Speech API token in API Key.
Click the "Save" icon.
Finally, you will see a success notification.

Evaluate the Benchmark of Each Chatbot Response

An information button is provided next to each response. Clicking it reveals detailed performance and inference statistics for that response, including token counts, processing speed, and computation time. This allows developers to monitor and optimize system performance in real time.

Using MCP tools

1. Verify the default MCP server in the GenAI-Chatbot tools

Click User.
Select Admin Panel.
Open the Tools section.
Under Manage Tool Server, ensure that file-utils / mcp-system-info are listed.
Click Save to apply the changes.

2. Open a new chat and select the tools

Select New Chat.
Click the “+” button in the chat window.
Choose the tool to open, e.g., system-info.
Verify that the tool icon appears next to the “+” button, with the number of icons corresponding to the number of selected tools.

3. Use the tool to obtain system information

Example: Ask the LLM model → “What is the current memory usage?”

4. Review the MCP server APIs and available functions

MCP - System-Info-Server: http://localhost:23952/mcp-system-info/docs

Using GenAI Assistant Mode

1. Configure Agent Settings

Click User.
Select Admin Panel.
Open the Agent section.
Select model.
Select tools.
Click Save to apply the changes.
Click Test button.

2. Start GenAI Assistant Mode

The GenAI Assistant Mode will appear, allowing you to invoke tools and start a conversation.

Change Suggest content

Change Suggested Content on the Agent Page.

Example

Creating an Audio + RAG Chatbot for Medication Assistant

1. Configuring TTS with Azure AI Speech API

(1). Create an Account on Azure AI Speech API

(2). Setting Up on GenAI Chatbot

2. Create a Knowledge

3. Create a Chatbot Assistant with RAG ( Knowledge )

4. Start a voice chatbot assistant with RAG

1. Make sure the select model is the RAG model.
2. Click the mic icon to start voice mode.
3. While speaking, the system will show a listening status.
4. After speaking, your question will appear as text.
5. The Medical Chatbot Assistant will response and play the answer with audio.

Add new mcp server on the Genai-Chatbot

Link

Difference between revisions of "Edge AI SDK/GenAIChatbot"

Latest revision as of 03:42, 26 September 2025

Contents

Introduction

How To

Download Models from Ollama (Support Device: Nvidia)

Delete Models on GenAI-Chatbot (Support Device: Nvidia)

Download SLM Models from GenAI Studio

Create a new Knowledge

Create a Chatbot Assistant with RAG ( Knowledge )

Configuring TTS with Azure AI Speech API

Create an Account on Azure AI Speech API

Setup the Azue Text-to-Speec in GenAI Chatbot

Evaluate the Benchmark of Each Chatbot Response

Using MCP tools

Using GenAI Assistant Mode

Example

Creating an Audio + RAG Chatbot for Medication Assistant

1. Configuring TTS with Azure AI Speech API

2. Create a Knowledge

3. Create a Chatbot Assistant with RAG ( Knowledge )

4. Start a voice chatbot assistant with RAG

Add new mcp server on the Genai-Chatbot

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools

@@ Line 1: / Line 1: @@
-  Brief of GenAI Chatbot
+= Introduction =
+<span style="font-size:larger;">GenAI Chatbot is a next-generation conversational AI assistant designed to provide natural, context-aware interactions. At its core, it utilizes efficient Small Language Models (SLMs) and supports the direct import of models fine-tuned within GenAI Studio, enabling easy deployment and immediate use of custom models in the chatbot.</span>
+<span style="font-size:larger;">The chatbot features advanced capabilities, including:</span>
+*<span style="font-size:larger;">Audio processing (Speech-to-Text [STT] and Text-to-Speech [TTS]).</span>
+*<span style="font-size:larger;">Retrieval-Augmented Generation (RAG).</span>
+*<span style="font-size:larger;">An embedded vector database (VectorDB).</span>
+*<span style="font-size:larger;">Model Context Protocol (MCP) tools, which enable AI agents to perform actions.</span>
+<span style="font-size:larger;">GenAI Chatbot features a flexible configuration suitable for diverse application scenarios and is optimized for embedded platforms such as the Ryzen AI 8000, NVIDIA Jetson Orin Nano, and Jetson Orin AGX.</span>
+= How To =
+== Download Models from Ollama '''(Support Device: Nvidia)''' ==
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ollama-1.png|none|900x600px|Ollama-1.png]]
+</div>
+*<span style="font-size:larger;">Go to the page [https://www.ollama.com/search https://www.ollama.com/search] and click on the "Models" tab as shown in Icon 1.</span>
+*<span style="font-size:larger;">Use the search bar shown in Icon 2 to find the model you want to download.</span>
+*<span style="font-size:larger;">In Icon 3, locate the name of the model.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+&nbsp;
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ollama-2.png|none|900x600px|Ollama-2.png]]
+</div>
+*<span style="font-size:larger;">After clicking in, you'll see the name and version of the model you need to download.</span>
+*<span style="font-size:larger;">Please note that the model size depends on your hardware resources.</span>
+*<span style="font-size:larger;">Make sure your hardware matches the specifications shown in [[#Platforms|the table above]].</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+&nbsp;
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ollama-3.png|none|900x600px|Ollama-3.png]]
+</div>
+*<span style="font-size:larger;">Go back to GenAI-Chatbot and create new chat window, as shown in Icon 1.</span>
+*<span style="font-size:larger;">Enter the model name you just saw into the search bar in the chat window, as shown in Icon 2.</span>
+*<span style="font-size:larger;">Then, click the download button at the location marked in Icon 3.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+&nbsp;
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ollama-4.png|none|900x600px|Ollama-4.png]]
+</div>
+*<span style="font-size:larger;">After clicking the download button, a notification will pop up.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+&nbsp;
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ollama-5.png|none|900x600px|Ollama-5.png]]
+</div>
+*<span style="font-size:larger;">After the download is complete, a notification will appear confirming the completion.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+&nbsp;
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ollama-6.png|none|900x600px|Ollama-6.png]]
+</div>
+*<span style="font-size:larger;">Next, you'll be able to find the model you downloaded in the model selection menu.</span>
+== Delete Models on GenAI-Chatbot '''(Support Device: Nvidia)''' ==
+*<div style="background-color:#fcfce6; padding:5px; margin:5px 0; font-size:larger;">⚠️ [https://ess-wiki.advantech.com.tw/view/Edge_AI_SDK/AI_Framework/AMD#Delete_model Delete Models For AMD]</div>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Delete-1.png|none|900x600px|Delete-1.png]]
+</div>
+*<span style="font-size:larger;">Click on icon 1 "User" to enter the Admin Panel (icon 2).</span>
+*<span style="font-size:larger;">After entering "Settings," click on icon 3 "Models."</span>
+*<span style="font-size:larger;">Click the download icon (icon 4) on the right.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Delete-2.png|none|900x600px|Delete-2.png]]
+</div>
+*<span style="font-size:larger;">After the window pops up, find the option to delete the model at the location of icon 1.</span>
+*<span style="font-size:larger;">After selecting the model you want to delete, click on icon 2 "Delete."</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Delete-3.png|none|900x600px|Delete-3.png]]
+</div>
+*<span style="font-size:larger;">Click "Confirm".</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Delete-4.png|none|900x600px|Delete-4.png]]
+</div>
+*<span style="font-size:larger;">A notification will pop up after a successful operation.</span>
+== Download SLM Models from GenAI Studio ==
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Genai-studio-1.png|none|900x600px|Genai-studio-1.png]]
+</div>
+*<span style="font-size:larger;">Click GenAI Studio Hub from the left menu.</span>
+*<span style="font-size:larger;">Enter the URL of your GenAI Studio.</span>
+*<span style="font-size:larger;">Click the "Save"&nbsp;</span>
+*<span style="font-size:larger;">Once the configuration is successful, a notification will appear as shown at icon 4.</span>
+*<span style="font-size:larger;">Displays a list of all models supported by GenAI Studio.</span>
+*<span style="font-size:larger;">Click on icon 6 to download the desired model.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+&nbsp;
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Genai-studio-3.png|none|900x600px|Genai-studio-3.png]]
+</div>
+*<span style="font-size:larger;">After the download is complete, a notification will appear, and the icon will change to a completion icon.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Genai-studio-4.png|none|900x600px|Genai-studio-4.png]]
+</div>
+*<span style="font-size:larger;">After the model has downloaded, you can select this LLM Model in new Chat&nbsp;</span>
+&nbsp;
+== Create a new Knowledge ==
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-1.png|none|900x600px|Rag-1.png]]
+</div>
+*<span style="font-size:larger;">Client the Workspace from the left menu.</span>
+*<span style="font-size:larger;">Go to the Knowledge.</span>
+*<span style="font-size:larger;">Click the + icon to add a new knowledge.</span>
+<span style="font-size:larger;">Here are sample&nbsp;files:</span> <span style="font-size:larger;">* PDF:&nbsp;[[Media:10_Essential_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.pdf|tial_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.pdf]]&nbsp;,</span> <span style="font-size:larger;">* Text:&nbsp;'''[[Media:10_Essential_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.txt|10_Essential_Q&As_About_Over-the-Counter_(OTC)_Medication_Use.txt]]''' &nbsp;</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-2.png|none|900x600px|Rag-2.png]]
+</div>
+*<span style="font-size:larger;">Enter the title.</span>
+*<span style="font-size:larger;">Enter the goal or description.</span>
+*<span style="font-size:larger;">Click the "Create Knowledge" button to finish.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-4.png|none|900x600px|Rag-4.png]]
+</div>
+*<span style="font-size:larger;">Click the "+" icon.</span>
+*<span style="font-size:larger;">Click the&nbsp;"Upload files" to upload files.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-5.png|none|900x600px|Rag-5.png]]
+</div>
+*<span style="font-size:larger;">Select the files you want to use.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-6.png|none|900x600px|Rag-6.png]]
+</div>
+*<span style="font-size:larger;">After the upload, the files will be displayed and a success notification will appear.</span>
+&nbsp;
+== Create a Chatbot Assistant with RAG ( Knowledge ) ==
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-7.png|none|900x600px|Rag-7.png]]
+</div>
+*<span style="font-size:larger;">Go to the Models tab.</span>
+*<span style="font-size:larger;">Click the "+" icon, to add a new model.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-8.png|none|900x600px|Rag-8.png]]
+</div>
+*<span style="font-size:larger;">On the add new model page, fill in and select the required fields shown in the red box:</span>
+**<span style="font-size:larger;">Click the "+" icon, to add a new model.</span>
+**<span style="font-size:larger;">* Title, * Subtitle, * Base Model, and * System Prompt.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-9.png|none|900x600px|Rag-9.png]]
+</div>
+<span style="font-size:larger;">Continuing from the previous page,</span>
+*<span style="font-size:larger;">Click the "Select Knowledge" to&nbsp;select the Knowledge you just created,</span>
+*<span style="font-size:larger;">Click the "Save & Create"&nbsp;to save and create.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-10.png|none|900x600px|Rag-10.png]]
+</div>
+*<span style="font-size:larger;">After successful creation, a notification will appear at icon 1.</span>
+*<span style="font-size:larger;">Then, click on the model ( icon 2 ) to enter the model chat.&nbsp;Start the Assistant chat.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-11.png|none|900x600px|Rag-11.png]]
+</div>
+*<span style="font-size:larger;">The chat window will display that the model in use is the Knowledge model you created.</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Rag-12.png|none|900x600px|Rag-12.png]]
+</div>
+*<span style="font-size:larger;">After starting the conversation, you will see that the model retrieves information from the Knowledge you created in its responses.</span>
+&nbsp;
+== Configuring TTS with&nbsp;Azure AI Speech API ==
+=== Create an Account on Azure AI Speech API ===
+<span style="font-size:larger;">'''1. [https://learn.microsoft.com/en-us/dotnet/azure/create-azure-account#option-2-sign-up-for-a-free-azure-account Create an Azure account]'''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-1.png|none|800x400px|Azure-1.png]]
+</div>
+*<span style="font-size:larger;">''Get started with Azure’s free account: new users receive $200 credit for 30 days and free access to popular services.''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-2.png|none|600x400px|Azure-2.png]]
+</div>
+*<span style="font-size:larger;">''AI Speech – Text-to-Speech: 500,000 neural characters per month for free accounts''</span>
-= Introduction =
+<span style="font-size:larger;">'''2. Set Up the Azure Speech Service'''</span>
-GenAI Chatbot is a next-generation conversational AI assistant built on the [https://ollama.com/search OLLAMA] architecture, supporting all models compatible with [https://ollama.com/search OLLAMA]. Designed for seamless integration with [https://genai-studio.gitbook.io/docs GenAI Studio], it allows users to directly import models that have been fine-tuned within [https://genai-studio.gitbook.io/docs GenAI Studio], enabling easy deployment and immediate use of custom models in the chatbot. At its core, GenAI Chatbot utilizes efficient Small Language Models (SLMs) to provide natural, context-aware interactions. The chatbot features advanced capabilities, including audio processing (Speech-to-Text [STT], Text-to-Speech [TTS]), Retrieval-Augmented Generation (RAG), and an embedded vector database (VectorDB), all within a flexible configuration suitable for diverse application scenarios. It is currently optimized for embedded platforms such as NVIDIA Jetson Orin Nano and Jetson Orin AGX.
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-1.png|none|900x600px|Azure-speech-1.png]]
+</div>
+*<span style="font-size:larger;">''In the [https://portal.azure.com/#home Azure Portal], click on "Create a resource".''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-2.png|none|900x600px|Azure-speech-2.png]]
+</div>
+*<span style="font-size:larger;">''Click on icon 1: "AI + Machine Learning"''</span>
+*<span style="font-size:larger;">''Then, click on icon 2: "Speech"''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-3.png|none|900x600px|Azure-speech-3.png]]
+</div>
+*<span style="font-size:larger;">''Click on “Start” at the position marked by the red box.''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-4.png|none|900x600px|Azure-speech-4.png]]
+</div>
+*<span style="font-size:larger;">''Click "Create" and fill in the necessary details:''</span>
+**<span style="font-size:larger;">''Subscription: Choose your Azure subscription.''</span>
+**<span style="font-size:larger;">''Resource Group: Select an existing group or create a new one.''</span>
+**<span style="font-size:larger;">''Region: Choose a region close to your location.''</span>
+**<span style="font-size:larger;">''Name: Provide a unique name for your Speech resource.''</span>
+**<span style="font-size:larger;">''Pricing Tier: Select Free F0.''</span>
+**<span style="font-size:larger;">''Click on icon 2: "Review + create"''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-5.png|none|900x600px|Azure-speech-5.png]]
+</div>
+*<span style="font-size:larger;">''After confirming the information, click the “Create” button.''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-6.png|none|900x600px|Azure-speech-6.png]]
+</div>
+*<span style="font-size:larger;">''After entering the overview page, click on the name link of the resource you created under "Resource."''</span>
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Azure-speech-7.png|none|900x600px|Azure-speech-7.png]]
+</div>
+*<span style="font-size:larger;">''In the left-hand menu, click on "Keys and Endpoint".''</span>
+*<span style="font-size:larger;">''Note down the Key1 or Key2 and the Endpoint URL; you'll need these to authenticate your API requests.''</span>
+&nbsp;
+=== <span style="font-size:larger;">Setup the Azue Text-to-Speec in&nbsp;GenAI Chatbot</span> ===
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Stt-tts-1.png|none|900x600px|Stt-tts-1.png]]
+</div>
+*<span style="font-size:larger;">Click the"Admin Panel."</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Stt-tts-2.png|none|900x600px|Stt-tts-2.png]]
+</div>
+*<span style="font-size:larger;">Click the Audio</span>
+*<span style="font-size:larger;">Select the "Text-to-Speech Engine" with&nbsp;"Azure AI Speech".</span>
+&nbsp;
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Stt-tts-3.png|none|900x600px|Stt-tts-3.png]]
+</div>
+*<span style="font-size:larger;">Enter the Azure AI Speech API token in API Key.</span>
+*<span style="font-size:larger;">Click the&nbsp;"Save" icon.</span>
+*<span style="font-size:larger;">Finally, you will see a success notification.</span>
+== Evaluate the Benchmark of Each Chatbot Response ==
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Chatbot-benchmark.png|none|900x600px|Chatbot-benchmark.png]]
+</div>
+*<span style="font-size:larger;">''An information button is provided next to each response. Clicking it reveals detailed performance and inference statistics for that response, including token counts, processing speed, and computation time. This allows developers to monitor and optimize system performance in real time.''</span>
+== Using MCP tools ==
+'''1. Verify the default MCP server in the GenAI-Chatbot tools'''
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Mcp-tool-1 1757558120013.png|900x600px|Mcp-tool-1 1757558120013.png]]
+</div>
+*<span style="font-size:larger;">Click '''User'''.</span>
+*<span style="font-size:larger;">Select '''Admin Panel'''.</span>
+*<span style="font-size:larger;">Open the '''Tools''' section.</span>
+*<span style="font-size:larger;">Under '''Manage Tool Server''', ensure that '''file-utils''' / '''mcp-system-info''' are listed.</span>
+*<span style="font-size:larger;">Click '''Save''' to apply the changes.</span>
+'''2. Open a new chat and select the tools'''
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Mcp-tool-2 1757558139296.png|900x600px|Mcp-tool-2 1757558139296.png]]
+</div>
+*<span style="font-size:larger;">Select '''New Chat'''.</span>
+*<span style="font-size:larger;">Click the “+” button in the chat window.</span>
+*<span style="font-size:larger;">Choose the tool to open, e.g., '''system-info'''.</span>
+*<span style="font-size:larger;">Verify that the tool icon appears next to the “+” button, with the number of icons corresponding to the number of selected tools.</span>
-== SLM Chatbot ==
+'''3. Use the tool to obtain system information'''
-The GenAI Chatbot utilizes a Small Language Model (SLM) as its core engine. SLMs provide efficient language understanding and generation capabilities, delivering fast and accurate responses with low computational overhead. This allows the chatbot to operate smoothly on both edge devices and server environments.
-== Integrate with [https://genai-studio.gitbook.io/docs GenAI Studio] ==
+*<span style="font-size:larger;">Example: Ask the LLM model → “'''What is the current memory usage?'''”</span>
-GenAI Chatbot is fully integrated with GenAI Studio, the AI LLM model management platform. Within [https://genai-studio.gitbook.io/docs GenAI Studio], users can:
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
-*Perform model fine-tuning.
+[[File:Mcp-tool-3 1756969033152.png|900x600px|Mcp-tool-3 1756969033152.png]]
-*Access a variety of model quantization methods for deployment on different hardware.
+</div>
-*Convert models easily to the formats required by various platforms.
+'''4. Review the MCP server APIs and available functions'''
-*Customize and personalize AI models with ease.
-This integration significantly simplifies the process of deploying and customizing chatbots for both enterprises and developers.
-== Audio ==
+*<span style="font-size:larger;">MCP - System-Info-Server: [http://localhost:23952/mcp-system-info/docs http://localhost:23952/mcp-system-info/docs]</span>
-*Speech-to-Text (STT)
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
-GenAI Chatbot supports natural voice input, automatically converting user speech queries into text for processing. The system can flexibly integrate with a variety of Speech-to-Text (STT) engines and providers. By default, it uses the '''Whisper-base''' model and supports integration with OpenAI API, Web Browser API, Deepgram API, and Azure AI Speech API, making it easy to connect to different STT services as needed.
+[[File:Image 1757556413952.png|900x600px|Image 1757556413952.png]]
+</div>
+== Using GenAI Assistant Mode ==
-*Text-to-Speech (TTS)
+'''1. Configure Agent Settings'''
-Chatbot responses can be automatically converted to natural-sounding speech using Text-to-Speech (TTS). Users are free to choose different voices and languages. The system supports the '''Web API''' by default and can be integrated with OpenAI API, Transformers, ElevenLabs, and Azure AI Speech API, providing flexible options for TTS services.
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
+[[File:Ai-agent-1 1757558170990.png|900x600px|Ai-agent-1 1757558170990.png]]
+</div>
+*<span style="font-size:larger;">Click '''User'''.</span>
+*<span style="font-size:larger;">Select '''Admin Panel'''.</span>
+*<span style="font-size:larger;">Open the '''Agent''' section.</span>
+*<span style="font-size:larger;">Select model.</span>
+*<span style="font-size:larger;">Select tools.</span>
+*<span style="font-size:larger;">Click '''Save''' to apply the changes.</span>
+*<span style="font-size:larger;">Click '''Test''' button.</span>
-== RAG ==
+'''2. Start GenAI Assistant Mode'''
-GenAI Chatbot supports Retrieval-Augmented Generation (RAG), combining large language models (LLMs) with external knowledge bases to greatly improve the accuracy and depth of answers. This makes it suitable for knowledge-based Q&A, document search, FAQ systems, and various information retrieval scenarios.
-*Embedded
+*The GenAI Assistant Mode will appear, allowing you to invoke tools and start a conversation.
-**The system converts various documents, knowledge sources, or custom content into vector (embedding) representations to enable efficient semantic search.
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
-**By default, the '''sentence-transformers/all-MiniLM-L6-v2''' model is used for embeddings, balancing computational efficiency and semantic understanding.
+[[File:Ai-agent-2 1757558303765.png|900x600px|Ai-agent-2 1757558303765.png]]
-**Supports automatic embedding of multilingual documents and multiple file formats (such as txt, pdf, markdown, html, etc.).
+</div> <div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
-**Custom embedding models are supported to flexibly meet the needs of different professional domains.
+[[File:Ai-agent-3 1757558242087.png|900x600px|Ai-agent-3 1757558242087.png]]
+</div>
+&nbsp;
-*VecorDB
+'''Change Suggest content'''
-**The generated embeddings are stored in a VectorDB (vector database), supporting large-scale, efficient semantic search and matching.
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">
-**By default, '''ChromaDB''' is used as the vector database, offering real-time retrieval and ease of deployment and management.
+[[File:Suggestions1.png|900x600px|Suggestions1.png]]
-**The system can be extended to support other mainstream VectorDBs, such as Pinecone, Weaviate, Qdrant, etc., making it suitable for enterprise or cloud-based deployments.
+</div>
-**Built-in features such as deduplication, sharding, and data weighting ensure query efficiency and result quality.
+*<span style="font-size:larger;">Change Suggested Content on the Agent Page.</span>
-= How To =
+= Example =
-*'''Download SLM Models from GenAI Studio'''
-*'''Crate a new chatbot assistant with RAG'''
-*'''Speak'''
-**Audio to Azure AI Speech API
-== Configuration ==
+== Creating an Audio + RAG Chatbot for Medication Assistant ==
- - Set Azure TTS Service
+<div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">[[File:EAS-GenAI DevKit.jpg|none|800x400px|GenAI Chatbot DevKit]]</div>
+=== <span style="font-size:large;">1. Configuring TTS with Azure AI Speech API</span> ===
+<div style="margin-left: 2em;"><span style="font-size:large;">[[#Create_an_Account_on_Azure_AI_Speech_API|(1). Create an Account on Azure AI Speech API]]</span></div> <div style="margin-left: 2em;"><span style="font-size:larger;">[[#Setup_the_Azue_Text-to-Speec_in.C2.A0GenAI_Chatbot|(2). Setting Up on GenAI Chatbot]]</span></div> <div style="margin-left: 2em;">&nbsp;</div>
+=== <span style="font-size:large;">2. [[#Create_a_new_Knowledge|Create a Knowledge]]</span> ===
+&nbsp;
+=== <span style="font-size:large;">3. [[#Create_a_Chatbot_Assistant_with_RAG_.28_Knowledge_.29|Create a Chatbot Assistant with RAG ( Knowledge )]]</span> ===
+&nbsp;
-[[Category:Editor]]
+=== <span style="font-size:large;">4. </span>Start a voice chatbot assistant with RAG ===
+<div style="margin-left: 2em;">
+&nbsp;
+</div> <div style="margin-left: 2em;"><div style="display:inline-block; box-shadow:0 4px 16px rgba(0,0,0,0.15); border-radius:12px;">[[File:Rag-stt-tts-4.png|none|900x600px|Rag-stt-tts-4.png]]</div>
+<span style="font-size: 15.6px;">1. Make sure the select model is the RAG model.</span><br/> <span style="font-size:larger;">2. Click the mic icon&nbsp;to start voice mode.</span><br/> <span style="font-size:larger;">3. While speaking, the system will show a listening status.</span><br/> <span style="font-size:larger;">4. After speaking, your question will appear as text.</span><br/> <span style="font-size:larger;">5. The Medical Chatbot Assistant will response and play the answer with audio.</span>
+</div>
+== Add new mcp server on the Genai-Chatbot&nbsp;==
+[https://github.com/ADVANTECH-Corp/EdgeAI_Workflow/blob/main/ai_system/amd/aimb-2210/create_mcp_server.md Link]<br/> &nbsp;