What Are Data Streams? A Simple Explanation

Rohit RaiRohit Rai
1 min read

Table of contents

what is stream of data ?

In the context of iterators, the term "stream of data" refers to a sequence of elements that can be accessed one at a time. An iterator doesn't store all elements in memory at once; instead, it generates or retrieves each element as needed, which can be particularly useful when dealing with large datasets or when elements are generated on-the-fly.

For example, imagine a long list of numbers that you want to process one by one. Instead of loading the entire list into memory, an iterator allows you to access each number individually, process it, and then move on to the next one. This approach minimizes memory usage and can improve efficiency, especially for large or potentially infinite sequences.

def generate_numbers(limit):
    num = 0
    while num < limit:
        yield num
        num += 1

number_stream = generate_numbers(5)
for number in number_stream:
    print(number)

Here, generate_numbers produces a "stream of data" by yielding each number one at a time. The data (numbers) are accessed sequentially without all being stored in memory at once.

1
Subscribe to my newsletter

Read articles from Rohit Rai directly inside your inbox. Subscribe to the newsletter, and don't miss out.

Written by

Rohit Rai
Rohit Rai

๐ŸŒŸ DevOps Engineer at IBM With a solid background in both on-premise and public cloud environments, I specialize in OpenStack, AWS, and IBM Cloud. My expertise spans across infrastructure provisioning using ๐Ÿ› ๏ธ Terraform, configuration management with ๐Ÿ”ง Ansible, and orchestration through โš™๏ธ Kubernetes and OpenShift. Iโ€™m skilled in scripting with ๐Ÿ Python and ๐Ÿ–ฅ๏ธ Bash, and have deep knowledge of Linux administration. Passionate about driving efficiency through automation, I strive to optimize and scale cloud operations seamlessly. ๐ŸŒ Additionally, I have a keen interest in data science and am excited about exploring how data-driven insights can further enhance cloud solutions and operational strategies. ๐Ÿ“Š๐Ÿ”