From RAG to Riches: AI That Knows Your Support Stack 

As large language models (LLMs) revolutionize enterprise productivity, one significant limitation remains: they can’t access your internal company data out of the box. That’s where Retrieval-Augmented Generation (RAG) steps in.

RAG extends the intelligence of LLMs by pairing them with a vector database that contains domain-specific knowledge—like your support tickets, Slack threads, engineering runbooks, Jira issues, and more. This enables LLMs to give grounded, relevant responses drawn directly from your own support ecosystem.

In this post, we walk through how you can build a RAG pipeline using YugabyteDB’s new vector capabilities to enable smarter and more context-aware support automation. Discover how to ingest internal documents, vectorize them, store them efficiently in YugabyteDB, and finally, how to use an LLM like GPT-4 to answer internal questions—rooted in your own support stack data.

Setup: YugabyteDB as a Vector Store 

In this blog we show you how to use YugabyteDB as a vector store. You can set up a YugabyteDB database locally by following these instructions. Use the v2.25 release (or higher if there is a new version available).

Make sure you have all the prerequisites from the install instructions. On a Mac, the install looks like this:

brew install wget 

wget https://software.yugabyte.com/releases/2.25.2.0/yugabyte-2.25.2.0-b359-darwin-arm64.tar.gz tar xvfz yugabyte-2.25.2.0-b359-darwin-arm64.tar.gz 

cd yugabyte-2.25.2.0/ 

./bin/yugabyted start

Once YugabyteDB is up and running, you can connect to the database with your favorite tool. YugabyteDB provides ysqlsh, which works just like psql.

After starting YugabyteDB, from the same directory, you can connect to the database by running

./bin/ysqlsh -h hostname -p 5433 -U yugabyte

As YugabyteDB is Postgres compatible, you can use psql. Install psql (on Mac) with:

brew install postgresql@15

Then connect with:

psql -h hostname -p 5433 -U yugabyte

Alternatively, to set up psql on EC2 (for example), install psql with:

sudo dnf install -y postgresql15-server postgresql15-contrib

Once connected to YugabyteDB, set up the vector extension, create a vector table, and create an index on it. Connect with:

psql -h 127.0.0.1 -p 5433 -U yugabyte

Then set up the vector extension and create a vectors table.

CREATE EXTENSION vector; 

CREATE TABLE vectors ( 

id TEXT PRIMARY KEY, 

article_text TEXT, 

embedding VECTOR(1536) 

); 

CREATE INDEX NONCONCURRENTLY ON vectors USING ybhnsw (embedding vector_cosine_ops);

To run the RAG application, follow these steps, making sure you’re using a Python 3.9 environment:

git clone https://github.com/kyle-hailey/rag_support cd rag_support 

python3.9 -m venv rag_support 

source rag_support/bin/activate 

pip install -r requirements.txt 

export OPENAI_API_KEY='your openAI key'

You need to have an OPENAI_API_KEY to run the python programs detailed in this blog. Make sure to set OPENAI_API_KEY in your environment. The OPENAI_API_KEY is read from the environment by the programs.

export OPENAI_API_KEY='your openAI key'

The rag_support repository has this structure:

rag_support/

├── data/ 

│ ├── github.txt 

│ ├── zendesk.txt 

│ ├── jira.txt 

│ └── slack.txt 

├── insert.py 

├── question.py 

└── requirements.txt

Once the GitHub repo has been cloned and set up, there are two steps to follow.

1. Create RAG data by vectorizing and loading files in ./data directory into YugabyteDB with insert.py
2. Ask questions to the LLM supplemented with RAG data using question.py

Both insert.py and questions.py are set up to connect to the locally installed YugabyteDB instance.

connection_string =
"postgresql://yugabyte:password@127.0.0.1:5433/yugabyte"

If you set up YugabyteDB on a remote machine, then this line will have to be modified to point to the other machine by replacing 127.0.0.1 with the remote IP.

STEP 1: Insert Vectorized Documents 

The first step is to vectorize support-related data using “insert.py”. The Python program insert.py will take all of the files in the ./data directory, vectorize them, and load them into the vector table in YugabyteDB.

% python insert.py
✅ Successfully connected to the database.
📄 Loading documents...
📦 Loaded 4 documents.
🔍 Vectorizing documents...
✅ Vectorization complete.

STEP 2: Ask Questions with Context 

Once the files from ./data have been vectorized and loaded we can start asking questions that will be supplemented with the information above using question.py. It executes these steps:

• Your question is converted to a vector
• The vector is used to retrieve relevant RAG chunks from the database • Those chunks are passed to GPT-4 in the context
• GPT-4 answers your question using that context

The questions.py shows the LLM response without the RAG data and with the RAG data. This highlights the power of the RAG data.

% python question.py
Ask me a question (press Ctrl+C to quit):
❓ Your question: Why is feature ABC-123 delayed
🔍 Retrieved context snippets:
- '### Source: Slack\n### Channel: #engineer' (distance: 0.1951)
- '### Source: Jira\n### Ticket ID: ABC-123\n' (distance: 0.2583)
- '### Source: Zendesk\n### Ticket ID: #789\n' (distance: 0.2948)
- '### Source: GitHub\n### PR: #456\n### Titl' (distance: 0.3059)
================================================================================
🤖 BASELINE ANSWER (without RAG context):
================================================================================
As an AI, I don't have specific information about a project or feature called ABC-123.
Delays in projects or features can be due to a variety of reasons such as technical
difficulties, changes in project scope, resource allocation, or unforeseen
circumstances. You may want to check with the project manager or team responsible for
more detailed information.
================================================================================
🧠 RAG-ENHANCED ANSWER (with context):
================================================================================
Feature ABC-123 is delayed because of two main issues. First, there is a missing OAuth
environment variable (`OAUTH_REDIRECT_URI`) on staging. Second, there is an unresolved
security review on the new login flow. The resolution of these issues is dependent on
the auth team and the security team.
================================================================================

Other questions you can try (by business function):

• Engineering/Dev
  • Why is feature ABC-123 delayed?
  • What’s the current status of the OAuth integration?
  • Is there any open security issue related to login?
• Developer Operations/SRE
  • Was there a deployment failure last week related to auth?
  • Who last worked on the login system?
• Support and Customer Context
  • Has Acme Corp had any recent issues with login?
  • What’s the resolution plan for Acme’s SSO problem?
• Documentation and Knowledge Base
  • How do I onboard a new OAuth provider?
  • What happened in the auth discussion thread last week?
• Meta Questions
  • Summarize everything blocking authentication features right now.

Why RAG Matters for Support Teams 

Traditional LLMs are smart, but they don’t know your company. RAG allows you to control the knowledge base, unlocking powerful use cases:

• Instant internal support without digging through old Slack threads
• Automated onboarding with access to team-specific docs
• Smarter incident retrospectives based on real historical artifacts

With YugabyteDB’s distributed vector store, your RAG setup is not only scalable, but fast and production-ready.

Next Steps 

You’ve just built a RAG pipeline tuned for your internal support stack. Want to scale it up?

• Schedule regular vector updates
  • Regularly replace the documents in ./data with new documents from
    • Zendesk
    • Jira
    • Slack
    • Github
  • And load these with insert.py to continuously enrich the RAG data
• Integrate into your Slack bot or support workflow
• Add chunking and metadata tagging
• Replace OPENAI with a local LLM like using Ollama to make sure all of your companies data stays inside the company’s wall

Conclusion

Building a RAG-powered support system with YugabyteDB transforms how your team accesses and leverages internal knowledge. Instead of hunting through scattered Slack threads, outdated wikis, and buried Jira tickets, your team can now ask natural language questions and get instant, contextual answers grounded in your actual support history.

The combination of YugabyteDB’s vector capabilities with modern LLMs creates a powerful foundation that scales with your organization. As your support data grows, the system becomes smarter and more valuable, turning your collective institutional knowledge into an always-available AI assistant.

Ready to turn your support stack into your competitive advantage? Start with the sample implementation above, then expand it to encompass your entire support ecosystem. Your future self—and your support team—will thank you!

Want to know (even) more? Explore how YugabyteDB can power the retrieval layer of a RAG pipeline—offering scale, resilience, and low-latency access to semantically rich data.