AI Career Reality Check · Skills
ChatGPT Skills Are Not AI Engineering Skills
A resume line that says "Expert in ChatGPT and prompt engineering" tells a hiring manager very little. Almost everyone applying for a tech role now uses chat assistants. What an AI engineering interviewer wants to know is different: can you build a system in which a model does a job thousands of times a day, and can you tell when it is doing that job badly? The distance between those two abilities is easiest to see as a ladder.
The skills ladder
| Level | What you can do | How you would show it |
|---|---|---|
| 1. Chat user | Ask clear questions, give context, iterate on answers, check facts | Better personal productivity; not usually a hiring signal on its own |
| 2. Skilled prompter | Write reusable prompts with examples, roles, output formats and constraints; know common failure modes | A prompt library for a real workflow, with before/after examples |
| 3. AI application builder | Call model APIs from code, parse structured output, handle errors, add retrieval over your own documents | A deployed app or API with source code |
| 4. AI engineer | Design systems with evaluation sets, monitoring, cost and latency control, security against prompt injection and data leaks | A project with an evaluation report: what you measured, what failed, what you changed |
Levels 1 and 2 are valuable for almost any job: marketing, support, analysis, teaching. Levels 3 and 4 are software engineering skills. The jump from 2 to 3 is where most people stop, because it requires programming.
What the jump looks like in practice
Suppose an accounts team wants invoice numbers, totals and GSTINs pulled out of supplier invoices. At level 2, you paste each invoice into a chat window with a good prompt and copy the answer. That works for ten invoices. It does not work for ten thousand, and nobody checks whether each answer is right.
At level 3, the model becomes one component in a program that checks its output. The model call is replaced with a stub here so the example runs without an API key:
import json
def call_model(prompt):
# stand-in for an LLM API call that was asked to return JSON
return '{"invoice_no": "INV-2041", "total": "18,450", "gstin": "07ABCDE1234F1Z5"}'
REQUIRED = {"invoice_no": str, "total": float, "gstin": str}
def extract(text):
raw = call_model(f"Extract invoice_no, total and gstin as JSON:\n{text}")
data = json.loads(raw)
errors = []
for field, kind in REQUIRED.items():
if field not in data:
errors.append(f"missing {field}")
elif kind is float:
try:
data[field] = float(str(data[field]).replace(",", ""))
except ValueError:
errors.append(f"{field} is not a number")
if len(data.get("gstin", "")) != 15:
errors.append("gstin must be 15 characters")
return data, errors
data, errors = extract("...invoice text...")
print(data)
print("errors:", errors or "none")Output:
{'invoice_no': 'INV-2041', 'total': 18450.0, 'gstin': '07ABCDE1234F1Z5'}
errors: noneNotice what the code does that a chat window cannot: it converts "18,450" into a number, enforces the 15-character GSTIN format, and returns a list of problems instead of trusting the model. A real version would also catch invalid JSON, retry or route failures to a person, and log every result.
Level 4 adds the questions an AI engineer is paid to answer: on a labelled set of 200 real invoices, how many fields come out right? Does accuracy drop for scanned invoices or for a particular supplier's format? What does each invoice cost to process, and how long does it take? What happens if an invoice contains text telling the model to ignore its instructions?
What to learn to climb the ladder
- Python, to a level where you can write functions, handle errors, work with JSON and call web APIs.
- Model APIs: sending messages, system instructions, structured or JSON output, token limits and pricing.
- Retrieval: embeddings, chunking documents, vector search, and the RAG pattern explained in RAG models in AI.
- Evaluation: building test sets, measuring accuracy, comparing prompt or model changes against a baseline.
- Engineering basics: Git, testing, deploying an API, logging, keeping keys and user data secure.
- Enough ML understanding to know why models make mistakes. What you actually need to learn about LLMs covers the essentials.
Presenting the skills honestly
- If you are at level 1 or 2, describe outcomes: "Built a prompt set that cut first-draft time for support replies; reviewed by the team lead". That is credible for non-engineering roles.
- If you are at level 3 or 4, link the repository and include the evaluation results, including what did not work.
- Avoid "AI expert" labels. Specific, checkable claims hold up better in interviews.
For project ideas at level 3, see a RAG customer support assistant project. For how AI engineering compares with machine learning engineering as a job, read AI engineer vs ML engineer.