Skip to content

Introduction on using a Vector Databases (RAG)

What is RAG and Document storage?

RAG (Retrieval Augmented Generation) is an AI architecture where a language model retrieves relevant information from an external knowledge base (like a VectorDB) before generating a response, improving the accuracy, informativeness, and factual correctness of the AI's output.

AI Models are based on out of date information, and do not have access to your private data. If you want them to have contextual access to your personal data or documents you can use a VectorDB.

Information

There are more options than just Vector databases for storing data, see other options that best suit your use case here.

What are the use cases for a Vector Database?

  • Enhanced Conversational AI: Powering chatbots and virtual assistants with access to a vast knowledge base, enabling them to provide more accurate, informative, and contextually relevant responses.
  • Personalized Recommendations: Delivering highly customized product or service recommendations by understanding user preferences and past behavior stored in a VectorDB.
  • Improved Search Functionality: Enabling semantic search capabilities, where users can find relevant information using natural language queries, going beyond traditional keyword matching.
  • Document Summarization and Question Answering: Quickly summarizing large volumes of text and accurately answering complex questions by retrieving and analyzing the most pertinent information from a VectorDB.
  • Anomaly Detection: Identifying unusual patterns and outliers in data by comparing real-time observations with historical data stored and analyzed within a VectorDB.

Privacy and your data

JellyFaaS offer the ability to store documents as part of your subscription. These are all stored and only accessible by your account, they are NOT accessible via anyone elses account. Alteratively you can use your own datasource and pass what in. All information in the REST Body is encrypted during transfer, and we do NOT store any data. The only data we store is around metrics (use, time running etc).

Information

Note: storing secrets within the platform is on the roadmap, so they will not need to be passed in as a request in the future.

Supported Database technology

We currently support MongoDB (more database support coming soon), if you want to use your own database this is easy to setup, and has a free tier for complete control of your data. If you want to setup a MongoDB please go here.

How VectorDB work

Vector Databases store data as a set of vectors, for example with a two dimensional X,Y vector you can have a coordinates of (5, 3), and with a three dimensional system X,Y,Z you can have (220, 32.4, 23). In AI's case, this can be several hundreds to thousands of coordinates per single vector!. JellyFaaS uses a standard of 768. What does this mean? Each word, sentence, paragraph is split into a vector and stored in the database. When you query using a Vector Database, it converts the query into a vector, compares and searches the Vector Database and then uses the results when querying the LLM (for example, ChatGPT or Gemini). It uses the data from the Vector DB to add context to your query from your data source.

For example, you have a company document with a policies and procedures in, you can store that document, and as queries against it. You can then add a simple chatbot interface and use it internally within your company or product.

Tutorial

In this example we are going to first query an existing free document, hosted in JellyFaaS, and then deploy a sample document, and query it.

The current document we are using is a dungeons and dragons campaign called the 'Frog Idol', just for context the Frog Idol is not a boss to be killed, but you need to 'appease the Frog Idol' to continue onwards with your journey.

Information

The Frog Idol is source from here, and was created is owned by Dyson Logo, please check out his work!

A Simple request

We have a Query, "What do I need to do, to appease the Frog Idol", posting this in ChatGPT or Gemini will produce something like this:

Gemini output
The concept of a "Frog Idol" and the rituals required to appease it are not part of any widely-recognized religion or mythology. It's possible that you might be referring to a fictional deity or a specific character from a book, movie, or game.

If you can provide more context about where you encountered the "Frog Idol," I might be able to give you a more specific answer.

Clearly, it doesn't know the correct answer, as that document was not part of the models training.

Using the following request to JellyFaaS:

curl --location 'https://ai.jellyfaas.com/query-service/v1/vectordb' --header 'jfwt: <token>' --header 'Content-Type: application/json' --data '{
    "query": "Tell me what to do to passify the Frog Idol.", 
    "vector_mongo_collection": "frogidol"
}'

and this time you get back something far more useful and accurate:

{
    "answer": "The Frog Idol desires the return of his lost treasures: an iron rose and a basket of amber. To appease him and obtain the amulet that will help you defeat the giants, you must find these objects and return them to him. \n\nHere's what you need to do:\n\n1. **Locate the Objects:**\n   * **Iron Rose:** The zombie master possesses the iron rose. You'll need to find his lair and defeat him to retrieve it. \n   * **Basket of Amber:** The nixies use the basket as a boat. Find their pool and try to bargain or fight to get the basket back. \n   * **Amber:** You'll need to find the amber within a swampy labyrinth dedicated to the Idol, which is located within the Black Mire.  Be careful, disturbing the amber will provoke the throgrin guarding it.\n\n2. **Consult the Oracle:** The Alabaster Oracle, the Frog God's wife, can provide information about the location of the lost treasures. You can find her in the Alabaster Cave.\n\n3. **Return the Treasures:** Once you have retrieved the iron rose, the basket of amber, and the amber from the labyrinth, return them to the Frog Idol in his location within the Black Mire. \n\n4. **Receive the Amulet:** After placing all three treasures in front of the idol, the Frog God will grant you a stone amulet in his shape, which will help you defeat the giants.\n\n**Important Considerations:**\n\n* **The Black Mire:** The Black Mire is a dangerous and hostile swamp. Be prepared for swamp encounters and challenging terrain.\n* **The Fruum:**  The Fruum, another god residing in the Black Mire, might interfere with your quest. Be prepared for a confrontation if necessary.\n\n**Remember:** This is just a general outline. You can adapt the quest to your own campaign and add your own twists and turns! \n",
    "query": "Tell me what to do to passify the Frog Idol.",
    "spanId": "1urV4UDNR",
    "messages": [
        "completed"
    ]
}

Uploading my own document

You can upload documents direct to your account (The exact number depends on your account level) using the following REST command:

Uploading a document into your environment
curl --location 'https://ai.jellyfaas.com/embedder-service/v1/upload?collection_name=test1' --header 'jfwt: <token>' --form 'file=@"<path to document>"'

this will return with the following information:

{
    "status": "chunking",
    "upload_id": "a892h3-3b7b-242s-24ms-978c685730a7"
}

This can take a while, you can check progress with:

Checking progress of a upload
curl --location --request GET 'https://ai.jellyfaas.com/embedder-service/v1/status?upload_id=a87e9aa3-3b7b-486a-baad-978c685730a7' --header 'jfwt: <token>' --form 'file=@"/path/to/file"'

This will return the following response:

{
    "collection_name": "test1",
    "embedded_chunks": 3,
    "status": "completed",
    "total_chunks": 3,
    "upload_completed": "Fri, 17 Jan 2025 16:00:13 GMT",
    "upload_id": "a892h3-3b7b-242s-24ms-978c685730a7",
    "upload_started": "Fri, 17 Jan 2025 16:00:11 GMT"
}

Once that has been uploaded, you can query the document with the previous command.

Using the SDK's

To do the same using the SDK's use the following code:

Other query options

REST Options, you can optionally set these when using the SDK too (see the SDK docs):

Key Required Description
query Y The query to ask
rag_query N Optional, you can ask the vector database a different query to the LLM, this may be useful if you are not seeing the correct results
vector_mongo_connection_string N Optional mongo db connection string (if you want to connect to your own mongo, ensure the IPS for google GCP are whitelisted!)
vector_mongo_database N Optional, the database name to use.
vector_mongo_collection Y The collection to use, that stores the embedded data
ai_platform N Optional of Gemini (default, part of the product, or you can use your own) or OpenAI ChatGPT.
api+key N If you want to bring your own Secret key (for Gemini, or ChatGTP [required])
structured_output N Base64 encoded JSON Schema representing the requested output model