Suppose you are given a .sql
file containing 10,000 lines of SQL commands and you were told to convert them to PySpark.
What software would you use?
...the task I want to accomplish:
- Somehow automate the creation of PySpark from a
.sql
file.
...your requirements for that task
- I get that there would most likely be an intermediary step between
.sql --> pyspark
, so the PySpark conversion is not a priority, the priority is having the.sql
file in some sort of 'universal' model.
...what you already know about software available for this purpose
- I have has a look at relational algebra, which looks like could be the 'universal` model, but I'm not sure if that would work. See: http://dbis-uibk.github.io/relax/calc/local/uibk/local/0
Any help on this would be great!