Pasted PDF Text Cleaner

A PDF stores glyphs positioned on a page, not paragraphs. When you select a passage and copy it, the viewer turns every visual line into a real line break, so the paragraph lands chopped every seventy characters. The hyphen the typesetter used to split a word at the end of a line comes along too, which is how a sentence about a compositor shows up as compo-sitor in the middle of your document.

The hard part is not joining lines, it is deciding which ones to join. Two signals do the work here: the punctuation ending the line, and its length compared with the median line of the document. A line ending in a period, a colon or a closing parenthesis closes the paragraph; a line much shorter than its neighbours is almost always the last of a block, a heading or a list item, so it stays put.

One surprise hides in the characters themselves: fi and fl often arrive as a single glyph, the typographic ligature the font uses to tuck the letters together. The text looks fine on screen, yet searching for fiscal in Word finds nothing and the spell checker flags the whole word. Journal headers repeated on every page and stray page numbers stuck inside a sentence get cleaned in the same pass.