DPUs: The Next Generation of Data Center Processors

When we first started having data centers, servers were simple. You had one CPU handling one server and that was that. Later on, virtualization came into play and many virtual servers resided on the same machine and shared resources including the CPU. A little later, other components of the network (ex. Routers) were virtualized on the same machine.

That is very convenient because instead of having multiple devices, we could have one powerful machine that virtualizes the whole network. However, what does that mean for the CPU?

The CPU -no matter how powerful- is designed for general purposes but now it's required to run very specific tasks like networking and cybersecurity which means the CPUs can be bottlenecks. That's where the DPU comes in.

What is a DPU?

The DPU or Data Processing Unit is a new class of programmable processor that combines 3 elements:

1- A high-performance CPU

2- A high-performance network interface

3- A set of flexible programmable acceleration engines designed specifically to improve data processing, networking, cybersecurity and others.

The DPU is basically a server integrated into the server as it has its own cache, CPU, ram, network interface, etc.

Why do we need DPUs?

As stated above, CPUs are not designed to handle the current workload and diversity applications require and they are struggling to keep up with tasks like AI and edge computing while doing their usual tasks like running the actual application logic.

A DPU offloads the CPU of those tasks and it is designed specifically for those tasks so it achieves 2 gains:

1- The networking, security, data processing and other tasks are done faster on the DPU than on the CPU and are done more efficiently

2- The CPU has more time to run the actual applications

By achieving both points, the overall server becomes faster, more secure and more power efficient.

Benefits of a DPU

1- Increased processing power: by offloading the CPU, more resources are available for application processing

2- Better performance: since the DPU is designed specifically for networking and data processing, it performs a lot better than CPUs

3- Reduced costs: by offloading the CPU, the hardware required is reduced and the power consumption is reduced by up to 30% according to NVIDIA.

Companies working on DPUs

An example of companies that are working on developing DPUs:

• NVIDIA: Their latest DPU called BlueField-3 which has 400Gb/s connectivity

• Microsoft: After acquiring Fungible earlier this year

• AMD: After acquiring Pensando last year

• Intel: Their product is called IPU but it is a DPU

Companies utilizing DPUs

Companies that are utilizing DPUs include:

• Cloudflare

• VMware

• Fortinet

Resources

• https://resources.nvidia.com/en-us-accelerated-networking-resource-library/datasheet-nvidia-bluefield

• https://resources.nvidia.com/en-us-accelerated-networking-resource-library/whats-a-dpu-data-product

• https://resources.nvidia.com/en-us-accelerated-networking-resource-library/nvidia-dpu-power-eff

• https://techcrunch.com/2023/01/09/microsoft-acquires-fungible-a-maker-of-data-processing-units-to-bolster-azure/

• https://www.amd.com/en/accelerators/pensando

• https://www.hpcwire.com/off-the-wire/amd-completes-acquisition-of-pensando/

• https://www.datacenterknowledge.com/data-center-faqs/data-processing-units-what-are-dpus-and-why-do-you-want-them

• https://en.wikipedia.org/wiki/Data_processing_unit

• https://www.youtube.com/watch?v=pQrM5L6C-FQ