Here is a piece of code to, first, make a list of your tags. If I have some time I will proceed next later ;).
list_of_tags = 
for filename in glog.glob("*_yourfiles.dat"):
with open(filename, 'r') as msg:
data = msg.read()
tags = re.sub(r"(?!tag[^\s]+\b)\b\w+","",data)
list_of_tags = list(set(list_of_tags.append))
You may need this list to proceed with
re.sub on each file between tag of interest and next tag. You remove all but not the blocks of interest (or the reverse and you match only groups of interest). And you append to file.
If you have a lot of data, using
re.sub to match is a must.
Well, here is what comes next, finally had time:
list_of_tags = ['[#tagA1]','[#tag2B]','[#tagC]']
string = "here is [#tagA1] and even more here with [#tag2B] why not of repeat of [#tagA1] and a last [#tagC] which is fine"
for tag in list_of_tags:
list_of_match = 
#print (string) #1
wstring = string.replace(tag,"__@_")
#print (wstring) #2
wstring = re.sub(r"(_@_[^\+)",r'\1'+'_@_',wstring)
#print (wstring) #3
for match in re.finditer(r"_@_[^@]+_@_",wstring):
with open(tag+"_outfile.dat", "a") as msg:
for match in list_of_match:
Note that the format of your tags is pretty horrible, you should not use special characters that may have to be escaped. Things like "@" or others unlikely combination to embed your tag would clearly be better.
String processing for first tag:
here is [#tagA1] and even more here with [#tag2B] why not of repeat of [#tagA1] and a last [#tagC] which is fine
here is __@_ and even more here with [#tag2B] why not of repeat of __@_ and a last [#tagC] which is fine
here is __@_ and even more here with _@_[#tag2B] why not of repeat of __@_ and a last _@_[#tagC] which is fine
So basically, you replace your tag by something you can work with easily (something generic/not this format).
Then, you add the same 'end' generic tag to your expression starting with the generic tag you just used to replace the tag of interest.
re.finditer() to get all expression between two tags with no "[" in between (cause you don't want to embed other tags in that).
Then, you fill a list with the matches. And you append to the corresponding file with the list content.
[#tagA1] _@_ and even more here with _@_
[#tagA1] _@_ and a last _@_
[#tag2B] _@_ why not of repeat of _@_
[#tagC] _@_ which is fine_@_
In 3 different files with terrible names:
$ cat '[#tagA1]_outfile.dat'
and even more here with
and a last
For sure you can use the same
for loop as seen in the first part, to loop over files and over tags, or the opposite, up to you.
Certainly not the best optimization overall but should still be pretty fast, with those very annoying tags that challenge my not too extended skills in regex.