Data Career · Git & GitHub Skills
One Skill That Changes Your Data Career: Git & GitHub
If your project folder has files called analysis_final.ipynb, analysis_final_v2.ipynb and analysis_final_USE_THIS.ipynb, you already know the problem Git solves. Version control keeps one copy of each file plus a full history of every change, who made it and why. For data work that matters more than it first appears: a wrong number in last month's report can be traced to the exact query change that caused it, and a model result can be reproduced from the code that produced it. This guide is the practical part: how to set up and use Git on a real data project.
Git and GitHub are different things
Git is the version control tool that runs on your computer. GitHub is a website that hosts Git repositories and adds collaboration features: pull requests, code review, issues and Actions for automation. GitLab, Bitbucket and Azure DevOps do similar jobs, and many Indian companies use them instead. The Git commands are the same everywhere.
A repo layout that works for data projects
sales-analysis/
├── README.md question, data source, how to run, findings
├── requirements.txt pinned Python packages
├── .gitignore
├── data/
│ ├── raw/ never edited; usually NOT committed
│ └── processed/ generated by code; NOT committed
├── notebooks/
│ └── 01-explore.ipynb
├── sql/
│ └── monthly_revenue.sql
├── src/
│ └── clean.py reusable functions moved out of notebooks
└── reports/
└── figures/
The rule behind it: commit the things that are hard to recreate (code, queries, documentation) and leave out things that code can regenerate or that should not be public (raw data, outputs, credentials).
What to keep out of Git
# .gitignore for a data project
data/
*.csv
*.parquet
*.xlsx
.env
.ipynb_checkpoints/
__pycache__/
models/*.pkl
.DS_Store
Three reasons to be strict here:
- Privacy. Customer phone numbers, names or transaction data must never go into a public repository. Even in a private company repo, data usually belongs in the warehouse or object storage, not in Git.
- Secrets. Database passwords and API keys go in a
.envfile that is ignored, and are read with something likeos.getenv("DB_PASSWORD"). If you ever commit a key, treat it as leaked and rotate it; deleting it in a later commit does not remove it from history. - Size. GitHub blocks files over 100 MB and warns well before that. For large datasets or model files, use Git LFS or a tool built for data versioning such as DVC, which stores the data elsewhere and keeps a small pointer file in Git.
The daily loop
git status # what changed?
git diff sql/monthly_revenue.sql # see the exact lines
git add sql/monthly_revenue.sql
git commit -m "Exclude cancelled orders from monthly revenue"
git push
Write commit messages that explain the business reason, not just "update query". Six months later, "Exclude cancelled orders from monthly revenue" tells a colleague why the March number changed. If you need a refresher on the basic commands, see Git commands every developer should know.
Notebooks in Git
A Jupyter notebook is a JSON file that stores outputs, images and execution counts along with the code. That makes diffs noisy and merges painful. Practical fixes:
- Clear outputs before committing, or install
nbstripout(pip install nbstripout, thennbstripout --installin the repo) so outputs are stripped automatically. - Use
nbdifffrom thenbdimepackage to see readable notebook diffs. - Move reusable logic into
.pyfiles undersrc/and import it. Plain Python files diff and review cleanly. - Keep notebooks numbered and focused:
01-explore,02-features,03-model.
One exception: for a portfolio repository, committing a notebook with outputs is useful, because GitHub renders it in the browser and reviewers can see your charts without running anything.
Branches and pull requests for data work
On a team, nobody pushes straight to main. You create a branch, make the change, and open a pull request (PR) so someone reviews it:
git switch -c fix/revenue-cancelled-orders
# edit, then add and commit as above
git push -u origin fix/revenue-cancelled-orders
# open a pull request on GitHub and request a review
For a SQL change, a good PR description says what changed, why, and how the result moved: for example, the old and new totals for one month, so the reviewer can check the effect. In analytics engineering teams that use dbt, every model change goes through this flow, and tests run automatically on the PR with GitHub Actions or a similar CI tool.
For ML work, the PR also records which experiment the code belongs to. Tag the commit that produced a model you ship (git tag model-v1.2) so you can always rebuild it.
Undoing mistakes safely
| Situation | Command |
|---|---|
| Discard uncommitted changes to one file | git restore notebooks/01-explore.ipynb |
| Unstage a file you added by mistake | git restore --staged data/raw/customers.csv |
| Fix the message of your last commit (not yet pushed) | git commit --amend -m "Better message" |
| Undo a commit that is already pushed | git revert <commit-id> (creates a new commit that reverses it) |
| See what a file looked like last month | git log -- sql/monthly_revenue.sql, then git show <commit-id>:sql/monthly_revenue.sql |
Avoid git reset --hard and force-pushing on shared branches until you fully understand them; they rewrite history that teammates may depend on.
How much Git each data role needs
- Data analyst: clone, commit, push, pull, a clean README. In Excel- and Power BI-only teams Git may not be used at all, but it still helps your portfolio. The guide on how Git and GitHub help you land a data analyst job covers the job-search side.
- Data scientist: branches, pull requests, resolving merge conflicts, notebooks handled as above, reproducible environments.
- ML or data engineer: everything above plus CI pipelines, tags and releases, and reviewing other people's code daily.
A weekend is enough for the basics; comfort comes from using it on every project. Start with GitHub basics for beginners if you have never created a repository, and use your next SQL practice from this step-by-step SQL plan as the first project you track in Git.
Frequently asked questions
Should I upload datasets to GitHub for my portfolio?
Only small, public datasets whose licence allows it, and say where they came from. For anything larger or private, link to the original source and include a script that downloads or prepares it.
Is Git used in Power BI or Excel work?
Less often. Some teams version Power BI project files or the SQL and Python behind reports, but many reporting teams use SharePoint or shared drives. Git matters most where code is written.
What is the difference between git pull and git fetch?
git fetch downloads new commits from the remote without changing your files. git pull fetches and then merges them into your current branch.