I'm looking for the best Service solution which would be able to trigger/monitor and control ETL process in BigQuery, then transfer files to Azure VM and do insert into MS SQL. Was trying to use Azure Data factory and create pipelines, but it only provides copy/move of the data from BigQuery which is not enough. Previously I was launching these commands from Ubuntu server(using GCloud SDK) and later on from Azure VM(windows server 2016) using batch scripts, but I would like some unified and lightweight solution. Anyone has any thoughts on this?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.