# In Short

I would like a software recommendation that summarizes a long text document and produces keywords for that text document. Ideally, it will exclude words like "with, and, the, etc." and include more important words. Ideally so i can script using Python3

I am unfamiliar with search engines but I know they have indexing techniques to rank keywords that summarize documents/webpages. My particular scenario does not require such a sophisticated suite (SQL databases and indexing entire filesystems, etc.). All my documents that I am interest to find are in one directory, so linux terminal utilities like grep are sufficient to parse. Maybe a full suite will perform better?

I have looked at PyLucene but it seems to have a lot more than I need. Can someone please give some recommendations for software or what kind of field of study this is, so I can do some of my own research, is it called indexing?

## Explanation of issue

I have hundreds of text documents (with a unique made-up file extension, .crp). Each of these .crp files describe, in plain text, a scenario for an experiment (basically a README) for the logs that my program generated.

For example I have directory called scanario_1 which contains a description.crp and .log files for the experiment. The description.crp describes the conditions under which the .log files were generated.

I write a each detailed description.crp file manually and also manually generate a keywords.crp file which summarizes the description.crp file. So I can perform queries on the keywords.crp files and not have to search the lengthy description.

Problem is, as I analyze the logs for performance and make notes on it (update the description.crt) I have to then go an update each keyword.crp file. And I would like to automate the creation of the keyword.crp files by having something read the description.crt file and produce the keywords