0

I have converted some PDF documents which contained JPEG into ASCII text.

Unfortunately, there are many
line
breaks in the middle of
sentences, leaving some lines with only a few words and the
text generally difficult to read.

Is there any
tool which will reformat the
text, taking into account sentences and
paragraphs
which might be broken up like
this. It should
recognize English sentence structure - that they generally
end with
period, question
mark full
stop, etc

2
  • 1
    What types of sentence structure (e.g. period) would you like to have a break in?
    – Tom
    Jan 2, 2016 at 23:17
  • I would like the program to understand the grammar of the English language. Jan 3, 2016 at 10:28

1 Answer 1

2

You can do this in most editing software (word processors/text editors) that lets you search/replace paragraph marks or line breaks. Here's is an example strategy described in Word syntax where ^p is a paragraph break, assuming no distinction between new-line characters and page breaks (which is the case for text files), and in which I will use _ for space characters.

First insert extra paragraph marks after sentence punctuation:

  • Replace . with .^p
  • Replace ? with ?^p
  • Replace ! with !^p

Remove leading and trailing spaces, keep repeating until no more found:

  • Replace _^p with ^p
  • Replace ^p_ with ^p

Reduce the number of consecutive paragraph breaks to two at the most. Keep repeating until no more found:

  • Replace ^p^p^p with ^p^p

Additional steps if you want to stitch sentences together into one parapraph:

  • Replace ^p^p with $$##$$ (or any string that is not in your document)
  • Replace ^p with _
  • Replace $$##$$ with ^p

Additional notes:

  • Put all this in a keyboard macro if you need to do it in multiple documents.
  • You may wish to reduce to one paragraph break only. Then you need a variation of this strategy.
  • You can use regular expressions as well, but I'm not fluent with them.
  • For that additional step your software must be able to handle very long lines, because you will have an intermediate result which is basically just one line.
  • Do all this on copies of your files so that you have a backup.
1
  • Much as I would prefer a program to do it, I had considered this, so +1. I would need to use a text editor which allows macros, so that I can store that sequence and repeat it for many files. Fortunately, NotePad++ will let me do so. Jan 11, 2016 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.