The short answer. For a column-aware diff — "which rows are new, which are removed, which changed" — the right tool is pandas with merge(..., how='outer', indicator=True). On the command line, Simon Willison's csv-diff takes a --key column and outputs added / removed / changed rows. For a one-off browser triage of a single column extracted from each CSV, paste the two columns into the compare tool — it shows in-both, A-only, B-only, union at a glance. Plain Unix diff is the fallback for files with stable row order. The decision table at the bottom maps your case to the right method, and the next section explains why "diff" alone usually isn't enough for CSVs.
Why CSV comparison needs more than plain diff
diff old.csv new.csv compares line by line. Two CSVs with identical content can still produce a "completely different" diff if:
- Row order changed. A re-export sorted differently — same data, but every line looks "moved".
- Column order changed. Someone added a column or rearranged. Now every row looks edited.
- Whitespace or quote style changed. One export uses
"alice", the otheralice. Same semantic content, diff sees them differently.
For data files, you usually care about row-level differences identified by a key column (email, ID, slug) — not line-level differences in the file representation. That requires a column-aware tool. The recipes below all assume you have such a key.
How do you compare two CSVs in pandas?
The data-team default. Five lines to get added, removed, and changed rows out:
import pandas as pd
old = pd.read_csv("old.csv")
new = pd.read_csv("new.csv")
merged = old.merge(new, how="outer", indicator=True)
added = merged[merged["_merge"] == "right_only"].drop(columns="_merge")
removed = merged[merged["_merge"] == "left_only"].drop(columns="_merge")
unchanged = merged[merged["_merge"] == "both"].drop(columns="_merge")
The indicator=True flag adds a _merge column that tags each row as left_only, right_only, or both. The outer join means every row from both files appears, not just the ones that match. Without on=, pandas joins on every shared column — useful for whole-row identity checks where any field difference makes the row "different". Specify on="email" (next snippet) when you have a real key column and want to detect changed fields within matching rows.
For "changed rows" specifically — same key, different values — merge on the key and compare:
combined = old.merge(new, on="email", how="outer", suffixes=("_old", "_new"), indicator=True)
changed = combined[
(combined["_merge"] == "both")
& (combined["name_old"] != combined["name_new"])
]
Specify the key column explicitly. Each non-key column gets _old and _new suffix so you can compare them directly. For a many-column CSV, build the comparison programmatically; for two or three columns, this hardcoded approach is the clearest.
How do you compare CSVs with csv-diff (Simon Willison's tool)?
The dedicated command-line tool, installable with pip install csv-diff:
csv-diff old.csv new.csv --key emailOutputs a human-readable summary: rows added, rows removed, rows changed (with field-level diff). For machine-readable JSON output for piping into a script:
csv-diff old.csv new.csv --key email --json
Supports CSV, TSV, and JSON inputs. The --key can be a comma-separated list for composite keys (--key "email,signup_date" for example). Used by Simon Willison's Datasette ecosystem; the de facto standard for "diff two CSVs with a key" on the CLI.
For multi-format support and a richer view, daff is the alternative: daff diff old.csv new.csv with HTML, ANSI-colour, and CSV output formats. Worth considering if you'll be sharing the diff with non-technical reviewers.
When is plain Unix diff enough?
When the two files have stable row order and identical column layout — usually because they're both exports from the same pipeline with the same sort. Then:
diff old.csv new.csvworks fine. Variants:
diff -u old.csv new.csv— unified format, common in git workflows.diff <(sort old.csv) <(sort new.csv)— sort both first, useful when order isn't stable but rows are identical otherwise. Bash-only syntax.comm -3 <(sort old.csv) <(sort new.csv)— only shows lines unique to either file (drops the common ones).
All of these compare lines, not rows-by-key. They work well for "two snapshots of a sorted log file" but break down on real CSVs with arbitrary row order. The CSV-aware tools above are usually the right answer.
How do you compare CSVs with SQL?
Load both into a SQLite database and use joins. Useful for ad-hoc analysis where you can write SQL faster than pandas.
sqlite3 compare.db << EOF
.mode csv
.import old.csv old
.import new.csv new
-- Added: in new, not in old (by email)
SELECT n.* FROM new n LEFT JOIN old o ON n.email = o.email WHERE o.email IS NULL;
-- Removed: in old, not in new
SELECT o.* FROM old o LEFT JOIN new n ON o.email = n.email WHERE n.email IS NULL;
.quit
EOF
For larger files, the same recipe in PostgreSQL or DuckDB — DuckDB has a read_csv() table function that reads CSVs directly without a separate import step, which is faster for one-off analysis.
When the browser tool fits (and when it doesn't)
The joinlines compare tool is line-based, not column-aware. It answers four questions on two lists at once — in both, A-only, B-only, union — with toggles for case-sensitivity and whitespace trimming. Two real cases where it's the right tool for CSV-comparison work:
- Extracting a key column from each file and comparing those. Open each CSV in Excel, copy the email column to clipboard, paste both into the compare tool. You instantly see which emails are added vs removed. Loses field-level changes, gains speed.
- Comparing two simple line-per-row files (a list of usernames, a list of URLs, a list of error fingerprints) where each row is one value. No key column needed because each line is the key.
Cases where the browser tool is the wrong tool:
- Multi-column row comparison. "Which rows have the same email but different status" can't be answered by a line-based comparator.
- Field-level change detection. Spotting that
name_old != name_newfor a given key needs pandas or csv-diff. - Quote-aware CSV parsing. If your CSV has commas in quoted fields, a line tool just sees them as different rows; pandas / csv-diff handle them correctly.
Honest summary: the browser tool is the right entry point for "I have two columns of values to compare". It's the wrong tool for "diff two real CSVs by key". The decision table below makes the line clear.
Which method should you use?
| Your situation | Best method |
|---|---|
| Two CSVs, multi-column, need row-level + field-level diff | pandas merge(... indicator=True) |
| Same as above, command-line / scripting context | csv-diff old.csv new.csv --key email |
| Need to share diff with non-technical reviewer | daff diff --output diff.html old.csv new.csv |
| Stable row order, identical column layout | Plain diff old.csv new.csv |
| Two columns extracted to clipboard, want which IDs added/removed | Browser tool |
| Two simple line-per-row lists (no CSV structure) | Browser tool directly |
| Composite key (email + date) | pandas merge(on=["email", "date"], ...) or csv-diff --key "email,date" |
| Multi-gigabyte CSV diff | DuckDB read_csv() + LEFT JOIN — streams, no memory ceiling |
Frequently asked questions
What's the right key column for a CSV diff?
Whatever uniquely identifies a row. For user data, usually email or user_id. For products, sku or product_id. For logs, often (timestamp, event_id) as a composite key. If you don't have a natural key, the comparison is "two unordered sets of rows" — in that case, set-based tools (comm on sorted files, or pandas with the full row as the key) make sense.
Why does csv-diff say "no changes" but the files look different?
Usually because the files differ in ways csv-diff doesn't care about: whitespace, quote style, column order, line endings. csv-diff parses CSVs semantically; if the parsed rows are equivalent, no diff is reported. Use plain diff if you want to see literal-text differences. Use the CSV cleanup recipe to normalise both files first if you want semantic-only diff with a paranoid sanity check.
How do I compare two CSVs in Excel?
Excel doesn't have a built-in CSV diff. Workarounds: (1) Use Power Query to merge both as tables with a Full Outer Join, then filter for rows where left or right is null. (2) Use the VLOOKUP or XLOOKUP recipe column-by-column — =IF(ISNA(XLOOKUP(A2, OldFile!A:A, OldFile!A:A)), "ADDED", "") for an "added" flag. Both work for small files; for > 50K rows, switch to pandas or csv-diff — Excel slows noticeably.
How do I detect which fields changed, not just which rows?
pandas merge with suffixes=("_old", "_new") as shown earlier, then column-by-column comparison. csv-diff outputs field-level changes natively in its JSON mode. For visual review, daff produces side-by-side coloured output — HTML mode is the most readable for a non-technical reviewer.
What if my CSV has multi-line cells (quoted fields with newlines)?
Line-based tools (diff, comm, browser compare) break on these — one logical row spans multiple file lines. Use only CSV-aware tools: pandas, csv-diff, daff, csvkit. The article on cleaning messy CSVs covers detecting multi-line cells before they cause downstream trouble.
Is it safe to use an online CSV diff tool for sensitive data?
Most "free online CSV diff" SaaS sites upload your files for server-side processing. For sensitive data (PII, financials, internal exports), use pandas / csv-diff / daff locally. The browser-based joinlines compare runs entirely in your tab — verifiable in DevTools → Network — but it only does line-based compare, not full CSV-aware row-by-key diff.
— the joinlines team, 2026-05-16