AI Testing Course Syllabus in Siliguri
The syllabus is broken down into 13 modules and totals roughly 60+ hours (indicative) of
learning. It follows the same flow as a real AI testing project: starting from GenAI
fundamentals, moving through prompt, LLM and hallucination testing, then into RAG and vector DB
testing, the RAGAS/LangSmith toolchain, and finally chatbot, agentic, MCP, AI security and
automation topics. Every module includes practical exercises with real tools, and the program
closes out with 5 guided projects.
Module 1: Generative AI Fundamentals — ~5 hrs
- LLM basics: architecture, parameters, context windows
- Tokenization, embeddings and model behaviour
- GenAI landscape: ChatGPT, Gemini, Claude
- How LLMs differ from traditional software
- Probabilistic outputs and non-determinism
Module 2: Prompt Testing — ~5 hrs
- Prompt design principles and patterns
- Evaluation of prompt outputs for correctness
- Edge cases, ambiguity and adversarial prompts
- Robustness testing across variations
- Prompt versioning and regression testing
Module 3: LLM Testing — ~5 hrs
- Output validation and consistency checks
- Bias detection and fairness testing
- Scoring approaches for LLM responses
- Automated LLM testing with Python
- Observability using LangSmith
Module 4: Hallucination Testing — ~4 hrs
- Types of hallucinations in LLMs
- Detection methods and fact-checking
- Mitigation strategies and grounding
- Using RAGAS for faithfulness scoring
- Reporting hallucination issues
Module 5: RAG Testing — ~6 hrs
- RAG architecture: retriever, generator, context
- Retrieval accuracy and relevance testing
- End-to-end RAG pipeline validation
- Chunking strategies and their impact
- Using LangChain for RAG test harnesses
Module 6: Vector Database Testing — ~5 hrs
- Vector databases: FAISS, Chroma, Pinecone
- Embedding quality and similarity search
- Index types and performance testing
- Retrieval accuracy measurement
- Data privacy and isolation testing
Module 7: RAGAS — ~4 hrs
- RAGAS metrics: faithfulness, answer relevance, context precision
- Building evaluation datasets
- Running RAGAS evaluations
- Interpreting RAGAS dashboards
- Integrating RAGAS into CI pipelines
Module 8: LangSmith — ~4 hrs
- LLM observability and tracing
- Debugging chains and agents
- Evaluating LLM outputs at scale
- Dataset creation and regression testing
- Monitoring production LLM applications
Module 9: Chatbot Testing — ~5 hrs
- Conversational flow testing
- Intent recognition and fallback handling
- Multi-turn context retention
- Testing for inappropriate responses
- Automating chatbot test suites
Module 10: Agentic AI Testing — ~5 hrs
- AI agents: planning, tool use, memory
- Multi-step agent validation
- Testing tool selection and execution
- Failure mode analysis for agents
- Using LangChain for agent test harnesses
Module 11: MCP Testing — ~4 hrs
- Model Context Protocol (MCP) overview
- Tool integration and schema validation
- Testing MCP servers and clients
- Security considerations for MCP
- End-to-end MCP workflow testing
Module 12: AI Security Testing — ~5 hrs
- Prompt injection and jailbreak testing
- Data leakage and privacy testing
- Adversarial inputs and robustness
- Testing guardrails and filters
- Reporting AI security vulnerabilities
Module 13: AI Testing Automation — ~5 hrs
- Playwright for AI application testing
- Selenium integration for legacy AI apps
- API testing with Postman for AI endpoints
- Python and JavaScript test automation
- Building CI pipelines for AI tests
| # |
Module |
Hours |
Tools |
| 1 |
Generative AI Fundamentals |
5 |
ChatGPT, Gemini, Claude |
| 2 |
Prompt Testing |
5 |
ChatGPT, Gemini, Claude |
| 3 |
LLM Testing |
5 |
Python, LangSmith |
| 4 |
Hallucination Testing |
4 |
RAGAS, LangSmith |
| 5 |
RAG Testing |
6 |
LangChain, RAGAS |
| 6 |
Vector Database Testing |
5 |
FAISS, Chroma, Pinecone |
| 7 |
RAGAS |
4 |
RAGAS |
| 8 |
LangSmith |
4 |
LangSmith |
| 9 |
Chatbot Testing |
5 |
ChatGPT, Python |
| 10 |
Agentic AI Testing |
5 |
LangChain, MCP |
| 11 |
MCP Testing |
4 |
MCP |
| 12 |
AI Security Testing |
5 |
Python, Postman |
| 13 |
AI Testing Automation |
5 |
Playwright, Selenium, Postman |
| Total |
62 Hours (~60+) |
10–12 tools |
Contacted by
UT
Mr. Fahim Abbas Zaidi.
Senior AI Testing Trainer
5+ Years Experience
Specialization: Manual & Automation
Testing