I have to process tons of text/html files. Im looking for a tool that helps me to create a graph with processing nodes in order to make transformations on this files. For instance:
- A node loads the file that have an specific exntension.
- This node is conected to other that checks if the file contains some word.
- Other node is able to receibe a file and transform it to a text stream,
- Other node is able to make some text replacement...
- etc...
What its the closest tool to do that?

  • Are you specifically trying to avoid programming in Python (for instance)?
    – Eric S
    Nov 3 '19 at 19:03

You are describing a type of process which the *nix toolchain design is suited: a suite of text-processing tools that each operate upon a file or text stream in sequence, each doing a simple operation and passing the output on to the next tool in the sequence.

grep is a program that is designed to be used in this way and that can be used to check whether a file contains some word or pattern.

There is no *nix tool that I know of that is specifically designed to strip HTML tags (if that's what you mean by "transform it to a text stream"), but you could easily create a simple tool to do this. See, for example, https://stackoverflow.com/questions/753052/strip-html-from-strings-in-python for suggestions on how to do this, or the strip_tags() function in PHP.

Depending on what the other steps are in your file processing workflow, you may find existing tools that do the job, or you may need to create some tools of your own.

The overall process may be controlled by a shell script (or batch file on Windows), particularly if you have conditions to evaluate and multiple paths that might be taken--for example, depending on whether or not a certain word is found.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.