AI Testing Course Syllabus in Asansol
Across 13 modules and about 60+ hours (indicative) of learning, the syllabus tracks how AI
testing is actually done on the job: starting with GenAI fundamentals, then moving into prompt,
LLM and hallucination testing, followed by RAG and vector DB testing, the RAGAS and LangSmith
toolchain, chatbot, agentic and MCP testing, AI security, and finally test automation for AI
systems. Practical exercises with real tools are built into every module, and the program closes
with 5 guided projects.
Module 1: Generative AI Fundamentals — ~5 hrs
- LLM basics: architecture, parameters, context windows
- Tokenization, embeddings and model behaviour
- GenAI landscape: ChatGPT, Gemini, Claude
- How LLMs differ from traditional software
- Probabilistic outputs and non-determinism
Module 2: Prompt Testing — ~5 hrs
- Prompt design principles and patterns
- Evaluation of prompt outputs for correctness
- Edge cases, ambiguity and adversarial prompts
- Robustness testing across variations
- Prompt versioning and regression testing
Module 3: LLM Testing — ~5 hrs
- Output validation and consistency checks
- Bias detection and fairness testing
- Scoring approaches for LLM responses
- Automated LLM testing with Python
- Observability using LangSmith
Module 4: Hallucination Testing — ~4 hrs
- Types of hallucinations in LLMs
- Detection methods and fact-checking
- Mitigation strategies and grounding
- Using RAGAS for faithfulness scoring
- Reporting hallucination issues
Module 5: RAG Testing — ~6 hrs
- RAG architecture: retriever, generator, context
- Retrieval accuracy and relevance testing
- End-to-end RAG pipeline validation
- Chunking strategies and their impact
- Using LangChain for RAG test harnesses
Module 6: Vector Database Testing — ~5 hrs
- Vector databases: FAISS, Chroma, Pinecone
- Embedding quality and similarity search
- Index types and performance testing
- Retrieval accuracy measurement
- Data privacy and isolation testing
Module 7: RAGAS — ~4 hrs
- RAGAS metrics: faithfulness, answer relevance, context precision
- Building evaluation datasets
- Running RAGAS evaluations
- Interpreting RAGAS dashboards
- Integrating RAGAS into CI pipelines
Module 8: LangSmith — ~4 hrs
- LLM observability and tracing
- Debugging chains and agents
- Evaluating LLM outputs at scale
- Dataset creation and regression testing
- Monitoring production LLM applications
Module 9: Chatbot Testing — ~5 hrs
- Conversational flow testing
- Intent recognition and fallback handling
- Multi-turn context retention
- Testing for inappropriate responses
- Automating chatbot test suites
Module 10: Agentic AI Testing — ~5 hrs
- AI agents: planning, tool use, memory
- Multi-step agent validation
- Testing tool selection and execution
- Failure mode analysis for agents
- Using LangChain for agent test harnesses
Module 11: MCP Testing — ~4 hrs
- Model Context Protocol (MCP) overview
- Tool integration and schema validation
- Testing MCP servers and clients
- Security considerations for MCP
- End-to-end MCP workflow testing
Module 12: AI Security Testing — ~5 hrs
- Prompt injection and jailbreak testing
- Data leakage and privacy testing
- Adversarial inputs and robustness
- Testing guardrails and filters
- Reporting AI security vulnerabilities
Module 13: AI Testing Automation — ~5 hrs
- Playwright for AI application testing
- Selenium integration for legacy AI apps
- API testing with Postman for AI endpoints
- Python and JavaScript test automation
- Building CI pipelines for AI tests
| # |
Module |
Hours |
Tools |
| 1 |
Generative AI Fundamentals |
5 |
ChatGPT, Gemini, Claude |
| 2 |
Prompt Testing |
5 |
ChatGPT, Gemini, Claude |
| 3 |
LLM Testing |
5 |
Python, LangSmith |
| 4 |
Hallucination Testing |
4 |
RAGAS, LangSmith |
| 5 |
RAG Testing |
6 |
LangChain, RAGAS |
| 6 |
Vector Database Testing |
5 |
FAISS, Chroma, Pinecone |
| 7 |
RAGAS |
4 |
RAGAS |
| 8 |
LangSmith |
4 |
LangSmith |
| 9 |
Chatbot Testing |
5 |
ChatGPT, Python |
| 10 |
Agentic AI Testing |
5 |
LangChain, MCP |
| 11 |
MCP Testing |
4 |
MCP |
| 12 |
AI Security Testing |
5 |
Python, Postman |
| 13 |
AI Testing Automation |
5 |
Playwright, Selenium, Postman |
| Total |
62 Hours (~60+) |
10–12 tools |
Contacted by
UT
Mr. Fahim Abbas Zaidi.
Senior AI Testing Trainer
5+ Years Experience
Specialization: Manual & Automation
Testing