Data Lake to SQL DB Data Movement ( or CSV to SQL Table data movement)
Arpit Tyagi
Step 1: Do we have the table employee in SQL?
Step 2: Use the dataset which is pointing to the file in Data Lake:
Step 3: Set up the Source and Sink in the copy activity:
Source - It will point to the dataset which is pointing to the employee file in data lake
Sink - We need to check the “Auto-Create Table option” as we want table to be created automatically. Write behavior would be “Insert“ and Sink side would be SQL Generic Dataset.
Step 4: Check again in SQL Database whether the table exists or not!
Employee table is present now and all the entries have been pushed.
Subscribe to my newsletter
Read articles from Arpit Tyagi directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Arpit Tyagi
Arpit Tyagi
Experienced Data Engineer passionate about building and optimizing data infrastructure to fuel powerful insights and decision-making. With a deep understanding of data pipelines, ETL processes, and cloud platforms, I specialize in transforming raw data into clean, structured datasets that empower analytics and machine learning applications. My expertise includes designing scalable architectures, managing large datasets, and ensuring data quality across the entire lifecycle. I thrive on solving complex data challenges using modern tools and technologies like Azure, Tableau, Alteryx, Spark. Through this blog, I aim to share best practices, tutorials, and industry insights to help fellow data engineers and enthusiasts master the art of building data-driven solutions.