I'm developing my own DPDK PMD for an ethernet device which receives/sends packets through PCIe. In order to develop my PMD, I'm using l2fwd application, which forwards packets from a physical port to my virtual one (which receives/sends packets through PCIe).

DPDK version - 20.11

Currently, I'm using only a single pair of TX/RX queues, so the application uses 2 lcores

• Lcore 0: RX port 0 TX port 1.
• Lcore 1: RX port 1 TX port 0.

Where port 0 is the physical one, and port 1 is the virtual one.

I send/receive packets using PCIe transactions (more than 1 transaction for every packet). So, in order to send a packet from the physical port through the PCIe I issue one transaction, wait until the transaction is completed and then issue another one (it needs to be serial).

The problem I'm facing is that it takes time to upload packets I receive on tx_pkt_burst callback (Lcore 0), and I don't want the core to be stuck in my tx_pkt_burst callback for a long time every time it is been called (the core is stuck in my code instead of polling the physical core for new packets).

In order to fix it, I store the packets received in tx_pkt_burst in an array, and upload the packets using the second core (Lcore 1). The second core polls my rx_pkt_burst callback, so I poll for new packets (and download them using PCIe), and upload packets received from tx_pkt_burst.

It goes something like that:

• Check for new packets to download.
• Start the download process if new packets are available.
• Check for packets to upload (received from tx_pkt_burst).
• Start the upload process if needed.

The code works, though I had to add spinlocks to some data since 2 cores will access the data.

I'm wondering if there is a cleaner way to handle this. Maybe a symmetric solution in which one core handles PORT0->PORT1 direction, and the second one handles PORT1->PORT0 direction.

I thought of using RTE alarms, something like that:

static bool alarm_set = false;

uint16_t my_eth_tx_burst(void *txq, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
{
  
  if (!alarm_set ) {
    rte_eal_alarm_set(1, my_alarm, NULL);
    alarm_set = true;
  }

 /* Save mbufs */
 /* Return number of saved mbufs */
}

static void my_alarm(void *param)
{
 /* Check for new packets to send */
 /* Check if any PCIe transaction completed, and start the next one if so */

 /* Call this function again in 1us */
 rte_eal_alarm_set(1, my_alarm, NULL);
}

That way, Lcore 0 will poll both the physical port for new packets, and my function to fire PCIe transactions.

What do you think? Is there a common solution to handle this kind of problems?

Thanks in advance