This is my first post, here it goes.
I'm working on a NLP/ML text-classification project.
I'm trying to create an architecture such that based on a particular input text, I (1) Classify it into various pre-determined classes, and (2) I want to figure out a relevant "score" for that particular input text.
In other words, given an input text (average: 6-10 sentence, 7-8 word), I classify it as (for example) 'Class A', and within that same classified text, I want to determine how relevant or irrelevant it is, based on 4 categories: positive words, negative words, relevant words, irrelevant words.
[Note: I have a lot of data for each each class (for text classification), along with data with its particular positive, negative, relevant, and irrelevant words (all separate and for finding relevancy).]
I got the text classification part by using a CNN approach, but can't seem to figure out how to determine the relevance.
Here's the caveat: Each class has different relevant, irrelevant, positive, negative pre-defined words and phrases of multiple length. I was thinking of using a TF-IDF algorithm, but I can't find a hybrid model which can take 4 metrics like the one I need.
Caveat 2: If the document has "positive" words, it should automatically be determined as relevant, however, if the number of "negative" words are more, we disregard it and only look at the "relevant" and "irrelevant" words.
num_positive > num_negative --> good num_positive <= num_negative --> check : [num_relevant and num_irrelevant:...]
With this "ultiamte relevancy" score, I check with a pre-defined score (pre-determined) and check to see if it is indeed greater than the pre-defined score, hence making it relevant for that case.
Is there any way to come up with a mathematical formula for getting the relevant score?
Can anyone give me any suggestions on how to go about this, to see if my logic is correct or not?
I don't know if I am asking for too much, so I apologize in advanced.
[Note 2: I'm doing this with python and Tensorflow]