Loading data efficiently into Snowflake is one of the most important aspects of building a scalable, cost-effective data platform. As Snowflake has matured, it now supports a variety of data loading options — each suited to different types of data, latency requirements, and workloads.

Choosing the best method isn’t simply a question of what’s newest or most powerful — it requires balancing several factors: performance, latency, cost, operational simplicity, and the skills available in your team.

In this article, we’ll explore the five main data loading methods in Snowflake. For each, we’ll examine how it works, where it fits best, and what to watch out for. To make things clearer, I’ll suggest diagrams you can use to break up the article visually — these are ideal for hand-drawing or recreating in your style.

Balancing Competing Priorities

Before diving into the options, it’s worth stepping back to consider the common trade-offs when loading data:

• Throughput — How much data can be loaded per unit time?

• Latency — How fresh is the data when it arrives?

• Cost — How efficiently are compute resources being used?

The diagram above illustrates that no loading method is perfect on all three fronts. As always in architecture, there are trade-offs. Most likely, the trade-off is between Throughput and Response Time, although it’s possible with either to deploy a system with very high costs.

Batch Loading: Reliable and Widely Used

COPY INTO on a Virtual Warehouse

For many Snowflake users, COPY INTO remains the most familiar method. It’s a classic batch process: files land in cloud storage, and a scheduled COPY command loads them into a target table.

How it works:

• Files (CSV, JSON, Parquet, etc.) are staged in an internal or external stage using a Cloud provider tool or the SnowSQL PUT command.

• A COPY INTO command runs on your own Virtual Warehouse.

• Files are ingested in parallel — one file per CPU core although a single load with mutiple files can use all the CPUs on a Virtual Warehouse.

The diagram above illustrates the overall process whereby a file from the Source System is PUT into Cloud Storage. The COPY command references a STAGE to point to the file, and using a FILE FORMAT, loads the data into a Snowflake table.

Strengths:

• Simple to set up — just SQL required.

• Full control over warehouse sizing and scheduling.

• Cost-effective for regular batch loads.

Considerations:

• Warehouses bill per 60-second minimum. If your job runs for 10 seconds, you still pay for 60.

• Parallelism depends on warehouse size: XSMALL = 8 CPUs = 8 files in parallel; MEDIUM = 32 CPUs = 32 files.

File size matters — for maximum throughput and cost-efficiency, aim for 100–250MB files.

The chart above illustrates how the COPY command scales to execute multiple parallel loads on a given warehouse. Effectively, as an XSMALL has 8 CPUs, it can load eight files in parallel, whereas an X-LARGE can load 128 in parallel.

COPY INTO with Serverless Tasks

Serverless Tasks offer an alternative to running COPY on your own warehouse:

How it works:

• You define a Serverless Task in Snowflake.

• The COPY INTO command is scheduled inside the Task.

• Compute is fully managed by Snowflake — no warehouse to size or manage.

The diagram above illustrates the same COPY process as above, but instead of using a Virtual Warehouse to perform the COPY, we deploy a serverless task.

Strengths:

• No need to manage or resize warehouses depending upon the number of files to load

• Compute is charged based upon the actual time (to the second) the task spent running which avoids waste from 60-second AUTO_SUSPEND time of the virtual warehouse and the minimum 60 seconds billing time.

• Unlike Virtual Warehouses which have a fixed size until altered, Snowflake provides workload management and the compute resources are automatically adjusted to the size of the workload.

• Very simple to operate.

Considerations:

• Serverless Tasks are billed at a 0.9 multiplier compared to the equivalent warehouse per-second charge. This means it’s 10% cheaper to execute loads using a serverless task than a virtual warehouse

• Although technically, the same file size rules apply (100-250MB per file), this is less of an issue when loading single files into a table as the billing is per second with no 60 second minimum charge.

Serverless Tasks are an entirely Snowflake native feature and need no external orchestration, for example using an ETL tool which simplifies the overall process.

Continuous Batch Loading

Snowpipe (Auto-Ingest)

Although Snowpipe is often referred to as a low latency method to load files, in reality the underlying technology is a COPY INTO command. The main difference between using the COPY method and Snowpipe is that Snowpipe can be configured to automatically load files as they arrive whereas COPY must be scheduled using either an external ETL tool or a Snowflake Task.

How it works:

• Files land in cloud storage (S3, Blob, GCS).

• A cloud event notification (SNS/SQS) on the Cloud Provider triggers Snowpipe.

• Snowpipe runs a predefined COPY INTO using serverless compute.

• Data is loaded within 60 seconds of file arrival with an average 30s execution time for a 100MB file.

The diagram above illustrates the overall process loading data files using Snowpipe which has the following stages:

1. Source files are delivered to Cloud Storage on an ad-hoc basis. This could be for example, data files generated from an IOT device or a custom application.

2. Immediately the file arrives, the cloud provider (AWS, Azure or Google) notifies Snowpipe providing the file name.

Snowpipe loads data files in parallel using Snowflake serverless compute.

Strengths:

• No warehouse required — the entire process is fully serverless.

• Load happens automatically, no scheduling or 3rd party orchestration tool required.

• Data files are automatically loaded immediately they are available instead of being batch loaded.

Considerations:

• Small files can become expensive: A common misunderstanding is that the Snowpipe charge of $0.06 per 1000 files processed is responsible, but in reality, the issue is the communication latency between Snowflake and cloud storage.

• 100–250MB files again strike the best balance between latency and cost.

• Requires some cloud setup (IAM permissions, event notifications), although this is a once-off operation.

• Be aware that Snowpipe loads data files in parallel; therefore, the load sequence is not guaranteed.

Snowpipe (Using REST API)

The diagram below illustrates this variation on the Snowpipe method described above and has exactly the same strengths and considerations.

The diagram above illustrates the REST API method to trigger Snowpipe and has the following steps:

1. The source systems deliver data files to cloud storage

2. The source systems use a REST API to trigger the load operation

3. Snowpipe loads the data in parallel

Additional Considerations:

• This method is designed primarily for situations where an existing near real-time load is migrated to Snowflake. Typically these use a REST API to trigger the load, and this method provides a simple migration approach.

Real-Time Snowpipe Streaming

Streaming Using Kafka

Designed for message based systems which deliver messages rather than files, this approach delivers sub-second latency and gigabyte throughput.

How it works:

• Source systems deliver messages to Kafka

• Kafka notifies Snowpipe Streaming

• Data is streamed row by row, directly into Snowflake tables.

• No files, no stages, no Snowpipe object — just pure streaming.

The diagram above illustrates this data flow whereby Kafka communicates directly with Snowpipe streaming to load data directly into Snowflake tables.

Strengths:

• Lowest latency — often sub-second.

• Most cost-efficient of all streaming options.

• No file management required.

• Throughput can reach extremely high rates (Gigabytes)

Considerations:

• Needs Kafka as a source of data as currently other message brokers are not supported.

Streaming Using a Client SDK

This method extends the Snowpipe Streaming solution to clients using other message brokers including custom built applications.

How it works:

• Source systems deliver messages to the Client Application

• The Client Application notifies Snowpipe Streaming using an SDK

• Data is streamed row by row, directly into Snowflake tables.

No files, no stages, no Snowpipe object — just pure streaming.

Considerations:

You are responsible for the message handling application and interface to the Java SDK which can be a high skill barrier for some Snowflake deployments

Data Loading Options Compared

The table below provides a quick summary comparing the data loading options, latency and cost.

Chosing the Optimum Data Loading Method

The diagram below illustrates a simple decision tree to help navigate the selection of the most appropriate Snowflake data loading method.

The decision is primarily based upon how data files are delivered.

1. File Based: means Snowpipe or Copy are the best options.

2. Message Based: Indicates Snowpipe Streaming using either Kafka or the Snowpipe Streaming SDK are the best options.

In terms of delivery method, this means:

1. Push: Means data files are delivered in an ad-hoc manner (for example from an IOT device) and ideally should be loaded without delay. In reality the load time from Snowpipe or Copy is exactly the same (the underlying technology is the same), but Snowpipe has the option to deliver files immediately upon arrival. Use the Auto-ingest or REST API as required. Both provide the same outcome.

2. Pull: Means data files are loaded on a regular batch schedule. In this case, using a COPY INTO command is the best approach using either an external orchestration tool (eg. Apache Airflow) or a serverless task.

In terms of batch data loading the options include:

1. Using Virtual Warehouse: Which was the traditional method used to load data using a COPY INTO command. This does however have limitations and potential risks arising from cost management. If this method is used, consult my article Best Practices for Using Bulk COPY to Load Data into Snowflake.

2. Using a Serverless Task: Using this method, we create a Snowflake Task to orchestrate data loading, and use Snowflake Serverless Compute resources. This has the advantage that Snowflake will manage the scale up and scale out processing, and given the 10% discount compared to a standard virtual warehouse, this is most likely to be the most cost efficient data loading method for batch loads.

Conclusion

Snowflake provides a highly flexible range of data loading options, from traditional batch COPY INTO commands to fully real-time streaming with Snowpipe Streaming. As we’ve seen, there is no single “best” method — the optimum approach depends on the type of data, latency requirements, and how the source systems deliver data.

For batch-based pipelines, COPY INTO remains a reliable choice, with Serverless Tasks offering a simple and often more cost-effective alternative to managing Virtual Warehouses.

For continuous or push-based delivery, Snowpipe is well-suited for loading files automatically with minimal delay, while Snowpipe Streaming offers the lowest latency and highest throughput for message-based systems — though with greater deployment complexity.

Ultimately, understanding the trade-offs of each method — particularly in terms of latency, cost, and operational effort — is key to selecting the right solution for your workload. In most cases, combining multiple approaches across different data domains will provide the best overall outcome.

Whichever method you choose, following Snowflake’s best practices — particularly around file sizing and warehouse/resource utilization — will help you achieve fast, reliable, and cost-effective data loading at scale.

Key Takeaways

• Snowflake offers five main data loading methods: COPY INTO, COPY with Serverless Tasks, Snowpipe (Auto-Ingest or REST API), Snowpipe Streaming via Kafka, and Snowpipe Streaming via Client SDK.

• The choice of loading method should be driven by latency requirements, source system behaviour (push vs pull), and team skills.

• For file-based batch loading, COPY INTO via Serverless Tasks is often the most cost-efficient and simplest to operate.

• For irregular or push-based file delivery, Snowpipe provides near real-time automated loading without the need for scheduling.

• For real-time, message-based streaming, Snowpipe Streaming delivers sub-second latency and high throughput, but requires greater technical investment.

• File size remains critical — 100–250MB is the optimal target for both COPY and Snowpipe methods to maximise parallelism and cost efficiency.

• Combining multiple loading methods within a Snowflake deployment is common and often provides the best results across different data domains.

Further Reading

• Best Practices for Using Bulk Copy to Load Data into Snowflake

• Best Practices for Reducing Snowflake Costs: Top 10 Strategies

• Boost Your Snowflake Query Performance With These 10 Tips

• Monitor Snowflake Usage & Cost

• How to Cut Snowflake Data Storage Costs with Zero Copy Clones

• Data Engineer’s Guide to Snowflake ETL Best Practices