Text Deduper

How to use it

1. Paste your text or list into the left panel - one item per line.
2. Choose how duplicates are compared: exact bytes by default, or ignore letter case, or ignore trailing whitespace.
3. Keep Keep first occurrence on to preserve the original order; switch it off to keep the last copy of each line instead.
4. Click Dedupe. The right panel shows the cleaned list, the box below lists the removed lines.
5. Read the counts in the status line, then Copy the result.

Common use cases

• Cleaning merged lists - combine several exported lists, then collapse the overlaps into one clean set.
• Keyword and tag hygiene - drop repeated tags that differ only by a trailing space or capitalization.
• Log triage - strip repeated identical lines to see which distinct events actually occurred.
• Shopping and todo lists - merge contributions from several people without losing who wrote what first.

Why it stays private

Deduplication is a set lookup inside your browser's memory. The list never leaves the page, so merged customer records or private notes stay on your device.

FAQ

Q: How are duplicates compared? A: Byte-for-byte by default. With Ignore letter case or Ignore trailing whitespace, the comparison normalizes those first, but the kept line is always the untouched original.

Q: What exactly does "keep last occurrence" do? A: For each group of duplicates, the final copy is kept at its own position and the earlier copies are listed as removed.

Q: Does it sort my lines? A: No. Order is preserved as pasted. If you need sorted output, run this first and paste the result into the line sorter.