The Future of Data Is IPFS + Pinata

The purpose of this article is to highlight the major problems with HTTP, explain how IPFS and decentralizing storage troubleshoot those, and then frame Pinata as the missing piece to the puzzle as the easiest, quickest and most trustworthy way to use IPFS.

Just as the tectonic plates of our physical world shift — the pieces of our digital infrastructure, too, adjust accordingly (albeit, much less forcefully). Although not as natural, the motions of the latter are as fated as the former and driven by humankind’s most outstanding quality: the drive to progress.

Today, when you upload content or interact with a platform — your data is funneled directly to a centralized hub. It simmers in a low-lit room with regimented rows of nodes that all rely on a single power source. Your data is no longer yours. It hangs by the noose of one switch, the possibility of a power outage, or even a single command from a company top dog.

The Inter-Planetary File System (IPFS) is "a distributed system for storing and accessing files, websites, applications, and data.". It is the skeleton for the majority of the non-fungible asset market and an infrastructural force to be reckoned with amongst future technologists.

In this article, I hope to answer the following questions:

What is IPFS? A summary.
Why I’m leaving HTTP behind. What’s not working?
How does IPFS work? Protocol architecture.
Benefits: How IPFS solves the pain points of HTTP.
Will IPFS save the world? Real-life examples of the power of IPFS.
Caveats: The Imperfections of IPFS — Could Pinata be a Panacea?

What is IPFS?

IPFS (InterPlanetary File System) is a distributed peer-to-peer file-sharing network. In addition to sounding like the newest Beastie Boys track, it’s also used to store and access any type of data online.

It was created in 2015 by Protocol Labs (founded by Juan Benet). The inspiration for a decentralized method of storing and accessing data came from precedents such as BitTorrent, git and other hypermedia data-sharing solutions.

IPFS is well-positioned to become the entire underpinning of the decentralized web by replacing HTTP, and ultimately upgrading the internet as we know it.

Why I’m Leaving HTTP

The past couple of decades has been HTTP’s heyday. It brought the Internet to life. Websites became dynamic and pleasant to scroll through. They also routed correctly and loaded with relative ease.

But alas, all good things come to an end and all great things must be replaced.

So, where is HTTP falling short?

The web of the future is data-heavy and media-rich. On-demand, high-definition video streaming and nebulous amounts of data are ubiquitous. HTTP was not designed for dynamism at such volumes. Link rot* and server crashes jeopardize the reliability of meaningful data and test our patience. On one hand, cloud computing has leveled up to compensate — but such a shift requires a change in the fundamentals. The infrastructure of the very way data is stored and distributed must adjust.

*link rot is when hyperlinks no longer point to their original content due to it being relocated to a new address or discontinued.

How Does IPFS Work?

In this section, we will deconstruct the architecture of the IPFS protocol.

At a high level, IPFS is a peer-to-peer network. Content is shared and made accessible through nodes all over the world.

Let’s break this down into three key elements:

Content Addressing: instead of location-based addressing (characteristic of HTTP).
The content is linked by Directed Acrylic Graphs (DAGs)
Content discovery is facilitated by Distributed Hash Tables (DHT).

Alright. Now let’s expand on these.

The current internet functions on location-based addressing. To navigate the internet as it is currently, you will search for the address. This is the domain name or even the IP address. Location-based addressing is stored in large centralized data farms. In the case the server fails, even momentarily, you won’t be able to find the data. While this data may live on someone’s computer - you cannot access it. IPFS uses content-based addressing. You’ll directly query the unique hash of the data or content. Whoever has that particular hash can then send it over to you in a jiffy.

Now that we’ve established how IPFS data is found, we can get slightly more granular with the data itself. Data is stored inside IPFS objects. Each object has a capacity of 256kb and often contains links to other IPFS objects. Simple text-only files can fit into one object. Anything larger is broken up into chunks of data. Each chunk is cryptographically hashed and assigned to a content identifier (CID). This content identifier contains all the information of your data at the given time.

Keeping the anatomy of IPFS data in mind, we now arrive at the method of discovery. The DHT is a 2-in-1 catalog and navigation system for IPFS. In the IPFS DHT, the key is the CID and the value is the set of nodes hosting the blocks. The DHT distributes this information in small tables throughout the network and coordinates the finding, sharing and fetching of data among nodes.

Benefits

The technological benefits of migrating the web to IPFS are more than meets the eye. The average user will likely not see any tangible benefits at first as the integration will be seamless. The potential of IPFS, arguably, has been mostly realized in the advent of NFTs. IPFS is the backbone of 80-90% of NFT-related projects such as marketplaces, games, and more.

Unsurprisingly, there are several overlapping values between IPFS and blockchain technology:

Verifiability + Immutability

Blockchain uses the ledger. IPFS uses content identifiers. Both are computed ways of keeping track of data changes. Every time data is updated or altered, a cryptographic hash will render a brand new CID. Instead of relying on centralized servers to ‘validate’ data authenticity, we can put our faith in a network of peers, supplemented by the objectivity of cryptography. Security is embedded in the architecture of IPFS itself.

Decentralization

The distributed nature of data on IPFS puts the power back into the hands of the people creating it. Centralized intermediaries no longer have the agency to censor data on a whim. The middleman is out. Your data now goes directly to its recipient. This goes beyond ownership, to reap financial benefits from cutting out extraneous fees. NFT marketplaces and decentralized exchanges have taken full advantage of this.

Reliability

Distributed data makes for a more robust internet. It saves the pain from server crashes or network attacks by bad actors today. Imagine the outrage that might arise from a popular blue chip NFT suddenly going offline — despicable.

Blockchain needs IPFS.

It’s not just an overlap in core values that explains IPFS’s central role in blockchain-based technology - it compensates for several of its key weaknesses.

Blockchain has many strengths, including immutable data and verification. However, the storage and distribution of big chunks of data are not one of them. It would be highly inefficient - not to mention costly - to do so on-chain.

Read more about how leading NFT marketplace Opensea leverages IPFS with Pinata here.

Will IPFS save the world?

IPFS upgrades the Internet, but does it upgrade society as a whole?

It is clear that IPFS revolutionizes digital infrastructure — but how does it relate to the rest of the world? Will there be a positive ripple effect of our changing digital realm? How does IPFS assuage the pain points of broader society?

A Hedge Against The Inevitable: Natural Disasters

The aftermath of natural disasters often destroys existing communication infrastructure. IPFS has the potential to assist relief efforts by providing a way to share information between peers. This could be for things like news, medical updates and contact details of emergency services.

A Tool for Democracy

With immutability and transparency, votes can be made and stored on IPFS without the risk of tampering. The leading voting tool for DAOs and other decentralized organizations, Snapshot, is an example of this.

Preserving History

IPFS’s versioning system allows you to update a file and link its CID to the previous version. This way, the entirety of history can be encoded and tracked. The Distributed Wikipedia Mirror by the IPFS team is a great example of information storage.

A Hedge Against Government Censorship

Centralization of the web puts individuals at risk of censorship. Consider political dissenters or journalists in areas with the restriction of freedom of the press. IPFS has already set a precedent for this. For example, when the Turkish government blocked access to Wikipedia in 2017, in the name of “national security,”. A group of vigilantes duplicated a copy of the Turkish Wikipedia and distributed it via IPFS. Another example of a populace leveraging IPFS for journalistic purposes is Myanmar in 2021.

Increase Global Access To The Web

2.7 billion still do not have access to the internet. Why? One culprit is geography. IPFS facilitates time-space convergence by increasing data distribution. Many developing nations have strong local networks, but diluted global networks. Furthermore, IPFS is programmed for efficiency. For instance, a peer-to-peer architecture may save up to 60% on bandwidth for video files. Saved bandwidth = saved costs = cheaper to access the web.

Caveats

Could Pinata be the Panacea to IPFS’ Imperfections?

So far, this article has evangelized IPFS. And rightly so. IPFS is an idyllic network, in theory. However, like most emergent technologies, it is not without imperfections. For it to be accessible and functional, it needs adjacent services like Pinata: the leading web3 media management service.

Let us elaborate on some ways Pinata patches up the gaps of IPFS:

Performance

As a peer-to-peer network, the performance of the IPFS network is affected by the number and quality of participating nodes. Not enough nodes will result in high latency and low content accessibility.

Pinata optimizes performance and provides certainty. Their robust content-delivery network (CDN) with over 200 edge-caching locations makes retrieving content lightning fast.

Not So Effortless File Hosting

By design, IPFS offers permanence and decentralized content. This is an ideological strength, but a structural weakness. There are three dangers arising from this. The first is the assumption of node reliability. Running an IPFS node absorbs significant bandwidth. This is not physically possible in certain geographic areas, not to mention plain undesirable. After all, without incentive — why would you volunteer to store thousands of peoples’ data on your computer? Secondly, although sparse, there is always the risk of nodes going offline (simultaneously). Content hosted on those nodes will be unavailable until another node takes the reins. Lastly, IPFS is optimized for efficiency. Its native ‘garbage collection’ process periodically empties cached storage to save node space.

Pinata’s pinning service stabilizes file hosting. Pinned content is automatically flagged as important and salvaged from the disposal. For more on this, read Pinata’s CEO Kyle Tut’s deep dive into why not all NFTs are the same.

Security

IPFS’s decentralized nature makes it prone to network attacks and malicious use of its networks. Further, due to the lack of built-in encryption or KYC mechanisms, it creates a security vector susceptible to man-in-the-middle attacks or DNS hijacking.

Pinata provides features that enhance the security layer and content privacy of the IPFS network. These include encryption, access controls, and content pinning.

Complexity

Getting started with IPFS is intricate and daunting for the less technical. Typically, users would need to spin up IFPS nodes and then wrap their heads around the complexities of content hosting.

Pinata simplifies this process with a user-friendly interface and API to easily upload, manage, and share content.

Conclusion

The IPFS protocol is one of many developments to materialize from the Internet’s recent seismic shifts. From its mass adoption, both technologists and the broader community will benefit. The decentralization of data improves data availability, security and integrity. In a world with data at the core — this is revolutionary. For technologists and builders specifically, IPFS’s peer-to-peer network increases content delivery speeds and reduces the need for bandwidth. Although not without its caveats, IPFS supplemented by adjacent services like Pinata, the curtain call of HTTP is nigh’, and the future of data for a more resilient, optimized internet is in clear sight.

Connect with me

Twitter (DMs open)

LinkedIn

YouTube

Sign up for a free Pinata account today\.*

Thanks for reading! 🫡

*Full disclosure: I am the developer evangelist at Pinata, so I breathe IPFS day-in day-out. This article conveys only my personal experience and opinion (backed by research, of course).

It's Time To Say Goodbye To HTTP.

Table of contents