Sort, Deduplicate and Clean Up Lines

How to use the line tools

1. Paste your text into Your lines, open a text file, or drop one onto the box.
2. Choose the options you need. The result updates straight away, and the counts show what changed.
3. Copy or download the result, or press Use as input to work on it again with different options.

Each line is treated as one item: a name, a URL, a row of a spreadsheet or a line of code. Typical jobs are removing duplicate email addresses, sorting a list alphabetically, tidying up text copied from a PDF, or turning a list into code by adding quotes and commas.

The order the options run in

Options always run in the same order, top to bottom, whichever you change first:

1. Whitespace: spaces are trimmed and repeated spaces collapsed.
2. Find and replace runs on each line.
3. Empty lines are removed, including any that the steps above emptied.
4. Duplicates are removed or picked out.
5. Sorting puts the remaining lines in order.
6. Prefixes, suffixes and numbers are added last, so numbering follows the final order.

For example, trimming before removing duplicates means “apple” and “apple ” count as the same line. If you need a different order, run one step, press Use as input, and run the next.

Rules

• Trimming removes spaces, tabs and other invisible spaces, such as non-breaking spaces, from the start or end of each line. Collapse turns runs of spaces and tabs inside a line into a single space.
• Find and replace changes every match on every line and ignores case unless you chooseMatch case. Plain text is matched exactly, so characters such as . and $have no special meaning.
• Regular expressions use JavaScript syntax. In the replacement, $1, $2and so on insert captured groups, $<name> inserts a named group, $& inserts the whole match and $$ inserts a dollar sign. ^ and $ match the start and end of each line, so ^ replaced with > quotes every line.
• Duplicates are compared after the steps above. The first copy is kept in its place.Only repeated lines lists each line that appears more than once, a single time. Letters with accents count as the same however they were typed, for example a precomposed “é” and “e” followed by a combining accent.
• A to Z follows the alphabetical rules of your browser’s language, so accented letters sort next to their plain letters. Upper and lower case sort together unless you choose Uppercase first. Lines that compare as equal keep their original order.
• Shortest first and Longest first count visible characters, so an emoji or an accented letter counts as one.
• Shuffle picks a new random order each time the text or options change, or when you press Shuffle again.
• Line breaks: Windows (CRLF), Mac and Linux (LF) and old Mac (CR) line breaks are all recognised. A line break at the very end of the text is kept, and does not count as an extra empty line.

Limitations

• The tool handles up to 200,000 lines or about 5 MB of text. Larger files are better handled by a spreadsheet or command-line tools such as sort and uniq.
• Find and replace works within single lines, so a pattern cannot match across a line break. To join or split lines, use a text editor.
• Some regular expressions, such as (a+)+$, can take a very long time on long lines and make the page unresponsive. If that happens, simplify the pattern or reload the page.
• Browsers store text boxes with Mac and Linux line breaks, so copied results use those. A downloaded file uses the same line breaks as the last file you opened. Files must be plain text in UTF-8, which almost all modern files are.
• Ignoring case uses general Unicode rules. A few languages have their own rules that are not applied: for example, Turkish dotted and dotless “i”.