RoCE (RDMA over Converged Ethernet)

View all platforms
FAQ

What is RoCE (RDMA over Converged Ethernet)?

RoCE (RDMA over Converged Ethernet) lets servers, storage systems and accelerators move data directly between memory locations across Ethernet. It reduces processor and operating system involvement, helping AI training, HPC and distributed storage transfer large datasets with lower latency and less CPU overhead.

1

Improves application efficiency

By offloading network processing, RoCE frees CPU resources for workloads and helps data move faster between systems.

2

Uses high-speed Ethernet

Organisations can support demanding data centre workloads without introducing a separate specialist interconnect.

3

Requires fabric design

Bandwidth, traffic prioritisation and congestion management must be configured correctly or packet loss can undermine performance.

The Role of RoCE in Modern IT Environments

RoCE enables enterprise compute, storage and accelerator platforms to exchange data with very low latency across a high-speed Ethernet network, supporting AI training, HPC and distributed storage workloads.

The fabric you choose determines how effectively you can prioritise traffic, control congestion and reduce processor involvement, so GPUs and CPUs spend less time waiting for data and more time running applications.

As a partner to vendors including Cisco, HPE Aruba, Juniper, and Arista, we specify RoCE-ready Ethernet fabrics against your requirements, so you're not overinvesting in bandwidth you won't use or exposing RDMA workloads to packet loss, delay and poor congestion handling.

How RoCE Works

RoCE moves data directly between the memory of RoCE-enabled systems over Ethernet. RDMA adapters handle the transfer, so the processor does not manage every step.

For IT teams

This cuts latency and reduces processor load. Teams can run high-performance traffic on a dedicated fabric or selected traffic class, and scale it in a controlled way across servers and switches.

The diagram below shows how roce works

Why Organisations Choose RoCE (RDMA over Converged Ethernet)

RoCE helps organisations move data faster across Ethernet, reduce latency and support efficient scaling for AI and high-performance compute environments.


Reducing Network Latency

RoCE transfers data directly between memory locations across Ethernet, reducing delays by bypassing much of the conventional network-processing path.

Increasing AI Infrastructure Performance

Faster data movement helps AI compute resources spend less time waiting for information, improving utilisation and shortening demanding workload processing times.

Minimising CPU Overhead

By reducing processor involvement in network transfers, RoCE leaves more CPU capacity for applications, calculations and business-critical processing.

Accelerating GPU Communications

In compatible environments, RoCE supports rapid data exchange between GPU-enabled servers, helping distributed training and accelerated computing workloads operate efficiently.

Supporting High-Performance Ethernet Fabrics

RoCE enables low-latency RDMA traffic across high-speed Ethernet without requiring a completely separate specialist network for demanding workloads.

Scaling AI Networking

A correctly designed RoCE fabric supports growing compute and GPU clusters while congestion management helps maintain consistent performance.

Common Enterprise Use Cases

RoCE supports low-latency, high-throughput data exchange across Ethernet for AI, storage, and compute workloads with demanding performance requirements.


Accelerating GPU Communications

Transfers data efficiently between compatible GPU servers, reducing host CPU dependency and improving distributed AI and accelerated computing performance.

Reducing AI Network Latency

Enables direct memory-to-memory transfers across Ethernet, cutting processing delays so latency-sensitive AI workloads exchange data more quickly.

Supporting AI Training Clusters

Connects GPU servers across clustered environments, helping training nodes synchronise model updates and process large datasets efficiently.

Enabling High-Performance Ethernet Fabrics

Supports low-latency RDMA traffic over Ethernet, using carefully configured congestion control and traffic management to maintain reliable performance.

Improving Distributed Storage Performance

Helps compatible storage platforms move data with less processing overhead, accelerating access to large, performance-sensitive enterprise datasets.

Scaling AI Infrastructure

Provides high-speed interconnects for expanding GPU clusters, helping communication performance keep pace as training and inference environments grow.

Key Considerations When Deploying RoCE (RDMA over Converged Ethernet)

Detailed fabric tuning prevents packet loss, congestion spreading, compatibility gaps and inconsistent performance across latency-sensitive AI and storage traffic.


01

Lossless Ethernet Configuration

Validate the selected RoCE mode and configure supported flow-control mechanisms consistently so packet loss does not trigger costly retransmission or workload stalls.

02

Network Congestion Management

Model traffic bursts and configure ECN, congestion notification and buffer thresholds where supported so hot spots do not spread through the fabric.

03

Switch Compatibility

Confirm switches, adapters, firmware and transceivers support the required RoCE version and features so end-to-end behaviour remains interoperable.

04

QoS Configuration

Map RoCE traffic to consistent priorities, queues and DSCP or PCP markings so lossless controls do not unintentionally affect ordinary Ethernet traffic.

05

AI Workload Optimisation

Measure message sizes, communication patterns and GPU utilisation under representative workloads so fabric settings address actual training or inference bottlenecks.

06

Fabric Scalability

Plan port speeds, topology, oversubscription and telemetry capacity so adding compute nodes does not create new congestion domains or unstable pause behaviour.

Technology Comparison: RoCE vs InfiniBand

Both options support low-latency RDMA, but they differ in fabric design, Ethernet integration, congestion control and operational ownership.

RoCE InfiniBand
Network fabric and transport model RoCE carries RDMA traffic across compatible Ethernet networks, using RoCEv2 where routable IP connectivity is required. InfiniBand is a purpose-built high-performance fabric with native RDMA transport, switching and fabric-management mechanisms.
Best-fit AI, HPC and storage workloads AI, HPC and storage environments that want RDMA performance while retaining Ethernet-based infrastructure, tools and operational skills. Dedicated AI and HPC clusters where tightly controlled low-latency communication and a specialised compute fabric are primary requirements.
Latency, bandwidth and congestion handling Delivers low latency and CPU offload but depends on carefully engineered congestion control, buffering and traffic prioritisation under sustained load. Provides native credit-based flow control and predictable fabric behaviour designed for intensive node-to-node communication at cluster scale.
Ethernet compatibility and operational requirements Requires compatible adapters, switches and firmware plus consistent ECN, QoS and, where used, priority flow-control configuration. Requires InfiniBand adapters, switches, cabling, subnet management and specialist operational knowledge separate from the general Ethernet estate.
What it is not built for Unmanaged or oversubscribed Ethernet networks that cannot provide the congestion control and end-to-end consistency required by RDMA traffic. General enterprise connectivity where broad Ethernet compatibility, conventional IP services and convergence with ordinary application traffic are required.

Enterprise Platforms We Recommend

Arista, HPE Aruba, Juniper, and Cisco platforms each suit different RoCE environments, operational models, and fabric requirements. Here's where each one fits best.


Arista Data Centre Switching product

Arista Data Centre Switching

Best for: Large cloud, AI, or financial-services estates with experienced automation teams needing carefully engineered low-loss Ethernet for RoCE-based HPC or distributed storage.

Strengths
  • Supports high-speed RoCE where selected models enable data-centre bridging
  • EOS supports QoS, ECN, and priority-flow controls for loss mitigation
  • ECMP and MLAG provide path resilience across leaf-spine RoCE fabrics
  • CloudVision workflows streamline high-volume RoCE fabric change control
View Arista Data Centre Switching › ›
HPE Aruba Data Centre Switching product

HPE Aruba Data Centre Switching

Best for: Mid-sized private-cloud estates with lean teams extending familiar Aruba operations into the data centre for carefully engineered low-loss Ethernet workloads.

Strengths
  • Supports high-speed RoCE where selected models enable data-centre bridging
  • AOS-CX supports QoS, ECN, and priority-flow controls where required
  • VSX and ECMP add resilient paths across leaf-spine fabric designs
  • Operational familiarity simplifies RoCE management for lean infrastructure teams
View HPE Aruba Data Centre Switching › ›
Juniper Data Centre Switching product

Juniper Data Centre Switching

Best for: Mid-sized and large data centres with automation-led teams prioritising intent-based fabric assurance for carefully engineered low-loss Ethernet supporting RoCE workloads.

Strengths
  • Supports high-speed RoCE where selected models enable data-centre bridging
  • Junos OS supports QoS, ECN, and priority-flow controls where required
  • ECMP and EVPN multihoming strengthen resilient RoCE fabric pathing
  • Apstra validates live state against intent to catch drift early
View Juniper QFX Platforms › ›
Cisco Data Centre Switching product

Cisco Data Centre Switching

Best for: Large Cisco-integrated data centres with dedicated fabric teams needing formal policy control and carefully engineered low-loss Ethernet for RoCE workloads.

Strengths
  • Supports high-speed RoCE where selected models enable data-centre bridging
  • NX-OS or ACI supports QoS, ECN, and priority-flow controls
  • ECMP and vPC provide resilient pathing across distributed RoCE fabrics
  • Nexus Dashboard centralises RoCE fabric policy, telemetry, and automation
View Cisco Nexus Platforms › ›
Steel City Consulting logo

Get a clear recommendation for your network

Unsure which platform is the right fit for your requirements? Our specialists can assess your workloads, existing estate, growth plans, and operational requirements, then recommend the right approach.
Get expert advice, no obligation

Related Technology Guides

RoCE performance depends on the transfer model, network fabric and accelerated systems around it; these guides provide the wider technical context.

RDMA

The direct memory-access process behind RoCE and why it reduces CPU involvement and transfer latency.

Read the guide

Spine-Leaf Architecture

Consider how equal-cost, high-bandwidth fabric paths support predictable communication between distributed compute nodes.

Read the guide

High-Performance Computing (HPC)

Where low-latency Ethernet communication supports tightly coupled compute, simulation and analytics workloads.

Read the guide

NVLink

Unpack how high-speed GPU interconnects inside a server complement RoCE communication between separate systems.

Read the guide

Related Technology Platforms

Explore the compute, storage and network platforms that combine to deliver low-latency RoCE communication for accelerated and data-intensive workloads.

AI Networking

Move latency-sensitive GPU and compute traffic across suitably configured, loss-aware high-speed Ethernet fabrics.

View AI Networking Platforms

Data Centre Switching

Supply high-bandwidth paths, traffic control and fabric resilience for RoCE communication between networked systems.

View Data Centre Switching

AI and GPU Servers

Operate distributed training and accelerated workloads that exchange model data directly between compatible systems.

View AI & GPU Servers

High-Performance SAN Storage

Supply compute nodes with large datasets through storage infrastructure designed for demanding throughput and latency requirements.

View High-Performance SAN Storage

Related Solutions

RoCE must be designed across compute and network layers so accelerated workloads receive the bandwidth, latency and traffic control they require.

Data Centre Modernisation Solutions

Successful RoCE deployment depends on switches, adapters, cabling, congestion controls and monitoring being validated together before latency-sensitive workloads go live.

Modernise Your Data Centre ›

AI Infrastructure Solutions

Distributed AI training benefits from an Ethernet fabric engineered to move model data between GPU systems with minimal delay, congestion and packet loss.

Design Your AI Infrastructure ›

Enterprise Compute Solutions

Expensive compute resources remain productive when RoCE-capable nodes are balanced against GPU density, storage throughput and available network capacity.

Plan Your Compute Environment ›

Networking Solutions

Performance-sensitive RoCE flows need defined traffic classes and proactive congestion monitoring within an architecture the network team can operate confidently.

Plan Your Network ›
Steel City Consulting logo
Need help implementing this technology?

Our specialists can recommend the right platform for your environment.

Speak to an expert

FAQ

What is RoCE and how does it work?

RoCE enables compatible servers, storage and accelerators to transfer data directly through application memory across a specially configured high-performance Ethernet network.

By bypassing several processor and software stages used in conventional networking, RoCE reduces delay and preserves CPU capacity for production workloads. This is most useful where distributed systems exchange large datasets frequently and network performance directly affects completion time.

Is RoCE the same as RDMA?

RDMA is the direct memory transfer method, while RoCE is a way of delivering that capability over Ethernet infrastructure.

This distinction matters because RDMA can also run over InfiniBand or iWARP, each with different infrastructure and management needs. Steel City Consulting can assess workload performance, existing networking and operational expertise, and whether Ethernet-based RoCE is the practical fit.

What is the difference between RoCE v1 and RoCE v2?

RoCE v1 is limited to a single Layer 2 network, while RoCE v2 can operate across routed Layer 3 networks.

RoCE v1 can suit contained environments where systems share the same network segment. RoCE v2 is usually the stronger choice for traffic moving across racks or routed boundaries, provided switches, adapters, firmware and congestion controls remain compatible throughout the path.

Does RoCE require a lossless Ethernet network?

RoCE requires a carefully engineered low-loss Ethernet network because dropped packets and congestion can materially reduce performance for latency-sensitive distributed workloads.

Depending on platform choice, the design may use priority flow control, explicit congestion notification or other congestion-management features. Steel City Consulting can configure and test the full path under realistic demand, reducing the risk of acceptable lab performance degrading in production.

Which workloads benefit most from RoCE?

RoCE benefits distributed AI, HPC and storage workloads most where frequent data transfers, network latency and processor overhead directly affect completion times.

Typical examples include GPU training clusters, parallel computing and compatible distributed storage. Standard office applications rarely gain enough value to justify the added network engineering, so workload profiling helps confirm whether RoCE is the right operational and commercial fit.

How does RoCE compare with InfiniBand?

RoCE delivers high-performance RDMA over engineered Ethernet, while InfiniBand uses a dedicated fabric designed specifically for demanding AI and HPC environments.

RoCE may suit organisations that want accelerated workloads aligned with established Ethernet operations, while InfiniBand offers a more specialised end-to-end architecture. Steel City Consulting can compare performance, scale, compatibility, skills and infrastructure costs to identify the stronger fit.

Get expert advice, with no obligation.

From new deployments to hardware refreshes and network reviews, we can help you identify what needs to change and how to move forward with confidence.
Contact us
A group discussing IT solutions
Can’t find what you’re looking for? We have more products available - just reach out to us directly! Found a better price elsewhere? Let us know - we’re happy to discuss price matching.