Which mode should I use for an email list?

Case-insensitive, in almost every practical case — real-world email systems overwhelmingly treat addresses as case-insensitive even though the technical spec allows case-sensitivity in the local part. Use exact-match only if you have a specific technical reason to preserve case-based distinctions.

Does this catch lines that are almost but not exactly the same?

No — it only removes lines that are character-for-character identical (or identical after lowercasing, in case-insensitive mode). A line with an extra space, typo, or slightly different wording is treated as a distinct entry, not a duplicate, since there's no reliable automatic way to define "close enough" for arbitrary text.

Which occurrence gets kept when a line repeats three or more times?

The first occurrence in the list is kept; every later repeat is removed, regardless of how many times the line appears total. The list order otherwise stays exactly as you pasted it.

Can I sort the list at the same time as removing duplicates?

Yes — /sort-lines/ has a dedicated dedupe-while-sorting mode that combines both operations in one pass. This tool is for when you specifically want to preserve the original list order and only remove repeats, without also alphabetizing.

← All Tools

Remove Duplicate Lines

Exact-match or case-insensitive duplicate detection for cleaning email and keyword lists.

This tool runs entirely in your browser. Nothing you type or paste here is ever sent to a server — see how the calculations work.

Paste a list, and this removes any line that's an exact repeat of one already seen earlier in the list, keeping the first occurrence and its original position. It's built for list-cleaning tasks — a collected email list with the same address entered twice, a keyword list scraped from multiple sources with overlaps, a to-do list copy-pasted from two different notes.

The deduplicated list appears here as you type.

Exact match vs. case-insensitive — the real trade-off

Exact-match mode treats two lines as duplicates only if they're character-for-character identical, including capitalization — "John@Example.com" and "john@example.com" would be kept as two separate lines in this mode, even though they refer to the same email address in practice (email addresses are technically case-sensitive in the local part per spec, though virtually no real-world mail provider treats them that way).

Case-insensitive mode treats two lines as duplicates if they match when both are lowercased, so "John@Example.com" and "john@example.com" would be recognized as the same entry and one removed — this is usually the more practically useful mode for real-world lists like email addresses, names, or keywords, where the same underlying entry might have been typed with different capitalization by different sources.

The trade-off: case-insensitive mode occasionally removes lines you actually wanted to keep distinct — for instance a list of proper nouns where "Apple" (the company) and "apple" (the fruit) are meaningfully different entries that happen to share the same letters. Choose exact-match if your list has this kind of legitimate case-based distinction; choose case-insensitive for typical human-entered lists like contacts or keywords where inconsistent capitalization is just noise, not signal.

Who uses this

Deduplicating a collected email list before an import, where the same address might have been entered with different capitalization.
Cleaning a keyword list scraped or compiled from multiple sources with overlapping entries.
Merging two to-do lists or reference lists and removing the lines that appear in both.

Edge cases to know about

This tool checks for exact-line duplicates (or case-insensitive-exact), not near-duplicates — a line with a trailing space, an extra period, or a slightly different spelling won't be caught as a duplicate of a very similar line. Run /remove-extra-spaces/ first if trailing whitespace is causing near-duplicate lines to be missed.
The tool keeps the first occurrence of each duplicate and discards later ones — if you need the last occurrence kept instead (for example, an updated entry appearing later in the list), you'd need to reverse the list order first.
Blank lines are treated as duplicates of each other by default (multiple blank lines collapse to none, or one, depending on your setting) since a run of blank lines in a list is rarely meaningful content — use /remove-empty-lines/ if you specifically want to control blank-line handling separately from duplicate removal.

Related tools

FAQ

Which mode should I use for an email list?: Case-insensitive, in almost every practical case — real-world email systems overwhelmingly treat addresses as case-insensitive even though the technical spec allows case-sensitivity in the local part. Use exact-match only if you have a specific technical reason to preserve case-based distinctions.
Does this catch lines that are almost but not exactly the same?: No — it only removes lines that are character-for-character identical (or identical after lowercasing, in case-insensitive mode). A line with an extra space, typo, or slightly different wording is treated as a distinct entry, not a duplicate, since there's no reliable automatic way to define "close enough" for arbitrary text.
Which occurrence gets kept when a line repeats three or more times?: The first occurrence in the list is kept; every later repeat is removed, regardless of how many times the line appears total. The list order otherwise stays exactly as you pasted it.
Can I sort the list at the same time as removing duplicates?: Yes — /sort-lines/ has a dedicated dedupe-while-sorting mode that combines both operations in one pass. This tool is for when you specifically want to preserve the original list order and only remove repeats, without also alphabetizing.