Sometimes I have to do complicated refactorings in a multi-language project. "Complicated" as in, far from stuff an LSP-server would allow, as a "rename an identifier". A recent example: find initialization of a specific array in a C file, for each element in it figure out if it's a string or an identifier (or combination of both), in case of identifier find its declaration in another file; then basically do some modifications across a few files based on the string/identifier naming pattern.
For C-language there's Coccinelle for that, but it is language-specific (so no cross-language refactorings) and is hard to debug too.
What I usually do is I write a parser which opens a file, goes through it line-by-line, trying to detect the pattern, while keeping the current state if pattern is multiline. So you basically get a state machine.
This is a monotonous work that I have to repeat, and I'd like to abstract it out. I'm wondering if there's any python module that, given a multiline pattern (perhaps in an EBNF grammar or something similar), would search for it in a file/text, and return all matches along with their locations.
What I tried so far:
- Lark. I was told on Gitter that it doesn't allow to just write the pattern to search for, instead it requires to have an EBNF that covers the whole file. This would be a lot of work. Even if I find an EBNF someone have already written, I feel like I could stumble upon corner-cases (such as syntax introduced in a newer language version or compiler-extensions); also, working with the AST of the whole file whereas I only need a specific portion of it sounds unwieldy.
- multiline regexp: I wasn't able to experiment with this too much to have a reasonable opinion. For starters, support is needed for both multiline matching and "nested structures" matching (e.g. balanced parentheses). regex python module is able to do both, but then I don't think it has a way to get "nth balanced expression". Substituting
nis a number results in exception when text has more similar balanced expressions with depth less than