AI Career Reality Check · Generative AI
RAG, Agents, APIs and Vector Databases Explained for Career Seekers
Picture a company with 2,000 pages of HR policies, product manuals and support tickets. Staff want to ask questions in plain English and get correct answers from those documents, not from the general internet. Building that system uses all four ideas in this article: an API to reach a language model, a vector database to find the right pages, RAG to combine the two, and, if the assistant must also take actions, an agent.
APIs: how your code talks to a language model
An API (application programming interface) is a defined way for one program to request something from another. With large language models (LLMs), your code sends an HTTP request containing a list of messages and settings, and the provider returns generated text. OpenAI, Anthropic, Google and others offer hosted APIs; open-weight models such as Llama or Mistral can also be run on your own servers behind a similar interface.
What you need to understand as a developer:
- Requests and responses in JSON, with an API key sent in a header. Keep the key in an environment variable, never in code.
- Tokens: models read and write in tokens (pieces of words). Cost and the context limit are both measured in tokens.
- Parameters such as temperature (more or less random output) and a maximum output length.
- Errors and limits: rate limits, timeouts and retries are part of everyday production work.
If REST APIs are new to you, practise on a simple public API first; this guide to fetching data from external APIs covers the basics.
Vector databases: search by meaning
A normal database finds rows that match exact values. A vector database finds items that are similar in meaning. It does this with embeddings: an embedding model turns a piece of text into a list of numbers (a vector) so that texts with similar meaning end up close together. "How many holidays do I get?" and "annual leave entitlement" share few words but produce nearby vectors.
The database stores those vectors and, given a query vector, quickly returns the nearest ones, usually by cosine similarity or a related distance. Dedicated options include Pinecone, Weaviate, Milvus, Qdrant and Chroma; PostgreSQL can do the same with the pgvector extension, and libraries like FAISS do it in memory.
The toy example below replaces a real embedding model with simple word counts so you can run it with plain Python and see the idea:
import math
from collections import Counter
docs = {
"leave-policy": "employees get 18 days of paid leave per year",
"wfh-policy": "employees may work from home two days a week",
"laptop-policy": "lost laptops must be reported to IT within 24 hours",
}
def embed(text):
return Counter(text.lower().split())
def cosine(a, b):
dot = sum(a[w] * b[w] for w in a)
norm = math.sqrt(sum(v * v for v in a.values())) * math.sqrt(sum(v * v for v in b.values()))
return dot / norm if norm else 0.0
question = "how many days of paid leave do employees get"
q = embed(question)
scores = sorted(((cosine(q, embed(t)), name) for name, t in docs.items()), reverse=True)
for score, name in scores:
print(f"{name:14} {score:.3f}")
best = scores[0][1]
prompt = f"Answer using only this context.\nContext: {docs[best]}\nQuestion: {question}"
print(prompt)Output:
leave-policy 0.667
wfh-policy 0.222
laptop-policy 0.000
Answer using only this context.
Context: employees get 18 days of paid leave per year
Question: how many days of paid leave do employees getThe leave policy shares six of the question's nine words, so it scores highest. Word counts fail on synonyms ("holidays" would not match "leave"), which is exactly the problem real embedding models solve. Searching each document is O(n) per query here; vector databases use approximate nearest-neighbour indexes such as HNSW so that search stays fast across millions of vectors.
RAG: retrieval-augmented generation
The last two lines of that output are RAG in miniature. Instead of asking the model to answer from memory, you retrieve the most relevant passages and put them into the prompt, then let the model generate an answer from that context. A production pipeline adds a few steps:
- Ingest: load documents (PDF, web pages, tickets) and split them into chunks of a few hundred tokens.
- Embed and store: create an embedding per chunk and save it with metadata (source, date, access rights).
- Retrieve: embed the user's question and fetch the top matching chunks, often combined with keyword search and a re-ranking step.
- Generate: send the question plus retrieved chunks to the LLM with instructions to answer only from them and cite sources.
- Evaluate: test with a fixed set of questions and check whether retrieval found the right chunk and whether the answer stayed faithful to it.
Most RAG quality problems come from steps 1 and 3 (bad chunking, wrong chunks retrieved), not from the model. For a deeper look, read RAG models explained.
AI agents: models that take steps
An agent is an LLM placed in a loop where it can decide to call tools. You describe the available tools (search the order database, send an email, run a calculation), and the model replies either with a final answer or with a request to call a tool with certain arguments. Your code runs the tool, sends the result back, and the loop continues.
messages = [user_question]
loop:
reply = llm(messages, tools=[lookup_order, issue_refund])
if reply asks for a tool:
result = run_tool(reply.tool_name, reply.arguments)
messages.append(result)
else:
return reply.textThat is pseudo-code, but every agent framework (LangChain and LangGraph, LlamaIndex, CrewAI, AutoGen or a provider's own SDK) is built around this loop. The hard parts are practical: limiting which actions an agent may take, asking a human to confirm risky steps, logging every tool call, and stopping loops that go nowhere. A RAG search is often one of the agent's tools.
How the four fit together
| Piece | Job | You would learn |
|---|---|---|
| LLM API | Generates text, follows instructions, calls tools | HTTP/JSON, tokens, error handling, cost control |
| Vector database | Finds relevant content by meaning | Embeddings, similarity, indexing, metadata filters |
| RAG | Grounds answers in your own data | Chunking, retrieval, prompts, evaluation |
| Agent | Plans and acts over several steps | Tool design, guardrails, logging |
A learning order that works
- Python basics, then call one LLM API and handle errors.
- Generate embeddings for a small document set and search them with Chroma or pgvector.
- Build a RAG assistant over documents you understand well, such as your college syllabus or a product manual, and write 20 test questions for it.
- Add one tool, such as a calculator or a database lookup, and turn it into a simple agent.
- Deploy it behind a small web API and log every request.
The RAG customer support assistant project is a good step 3, and LangGraph agent workflows helps with step 4. For what to learn about the models themselves, see what you actually need to learn about LLMs.