QEMU vhost-user-rdma - RDMA emulation

Background

What follows builds on the material presented in vhost-user back ends; it should be reviewed before moving forward with the content in this file.

Description

The vhost-user-rdma device provides a virtio-rdma front-end in QEMU and forwards virtqueue handling to an external vhost-user back-end. It is intended for test and development environments where the RDMA control plane and data plane run outside QEMU itself.

QEMU implements the guest-visible VirtIO device, configuration space and vhost-user connection management. The external back-end is responsible for implementing the RDMA device model, queue processing and any host networking or RDMA transport used to exchange packets with peers.

The PCI device is modern-only. It is exposed as vhost-user-rdma-pci for compatibility with existing command lines and as vhost-user-rdma-pci-non-transitional for explicit modern-only use.

Properties

The device requires a chardev that connects to the vhost-user back-end. It also exposes helper properties for queue layout and host pairing:

  • num-queues controls the total number of virtqueues.

  • data-queue-pairs controls how many data queue pairs are advertised.

  • paired-netdev points at a virtio-net device whose MAC address is copied into the RDMA config space.

  • paired-net-mac can be used to provide the MAC address directly.

Examples

Start the external RDMA back-end first. The binary name and accepted options depend on the implementation used for the vhost-user side; the command below shows a DPDK-based back-end as an example:

host# dpdk-vhost-rdma --socket-path=/var/run/rdma.sock

Then start QEMU with a matching chardev and memory backend:

host# qemu-system-x86_64                                       \
    -m 4096                                                    \
    -object memory-backend-memfd,id=mem,size=4G,share=on      \
    -numa node,memdev=mem                                      \
    -chardev socket,id=rdma0,path=/var/run/rdma.sock          \
    -device vhost-user-rdma-pci,chardev=rdma0,paired-net-mac=52:54:00:12:34:56