Network traffic forwarding in Linux Kernel

When developing virtual network device drivers for Linux, sometimes there is a need of redirecting packets between the real device and a virtual one.
Fortunately, the Linux kernel (version 4.14 in our case) has the right tools to do so, without any extra headache. Let’s look at redirecting incoming packets from a real network device. To do so, you don’t even have to change the sources of a real network driver itself. All you need to do is just use the mechanism of rx_handlers. To add your own handler, you need to use the following API call from your virtual driver:

int netdev_rx_handler_register (

struct net_device * dev,

 rx_handler_func_t * rx_handler,

 void * rx_handler_data);

The signature of a rx_handler is following:

typedef rx_handler_result_t rx_handler_func_t(struct sk_buff **pskb);

Here is the list of possible return codes:

enum rx_handler_result {
RX_HANDLER_CONSUMED,
RX_HANDLER_ANOTHER,
RX_HANDLER_EXACT,
RX_HANDLER_PASS,
};

This enum has a good amount of description in Linux sources, so we will stop in what we need – RX_HANDLER_ANOTHER. So, let’s summarize what we should do to redirect incoming packets to a different device:

1. Register rx_handler
2. In rx_handler change dev (pointer to net_device struct) to the device we want to redirect packets to
3. Return RX_HANDLER_ANOTHER to inform the kernel that we actually changed the destination device

Here is a simplified code of the handler:

static rx_handler_result_t handle_frame(struct sk_buff **pskb)
{
    struct sk_buff *skb = *pskb;
     // virtnet_dev is a pointer to net_device struct
// you can obtain the pointer with this call 
// dev_get_by_name(&init_net,device_name_cstr);
skb->dev = virtnet_dev;
return RX_HANDLER_ANOTHER;    
}

That’s it, we have a working solution for the problem. However, this type of solution comes with its own limitations. We cannot add a network device with a registered rx_handler to a virtual bridge. This is a known issue and it happens because bridge code uses the same API and tries to register rx_handler (but devices can have only one rx_handler registered!).

In our case, it wasn’t a problem, because in our topology only virtual devices were enslaved by a bridge. In the other case, it can be achieved by using protocol handlers and dev_forward_skb call, but this is a different subject to describe.

Another use case is redirecting all outgoing packets from the virtual device and sending them with a real one.

All network devices have their ndo_start_xmit callback. There we already have fully-formed packets, which are ready to be sent. Here is a simplified example of how to send all packets through a different network interface:

static netdev_tx_t virtnet_xmit(struct sk_buff *skb, struct net_device *dev)
{
// eth_dev is a pointer to net_device struct
skb->dev = eth_dev;
dev_queue_xmit(skb);
return NETDEV_TX_OK;
}

As you can see, it’s enough to change skb->dev and call dev_queue_xmit(skb)!

You shouldn’t even call free() for skb, it’s already implemented in dev_queue_xmit logic.

Need help with Linux development?