Re: Questions on network and block device related IRQ handling

[Shane Wayne <zx0319 at gmail dot com>]

Thanks William, your interpretation really helps me a lot, after
viewing all the refs you mentioned apart from this I also have done
some search on NAPI, this term makes me confused:
1, in the ref:https://web.archive.org/web/20111030030517/http://www.linuxfoundation.org/collaborate/workgroups/networking/networkoverview
it says that: "This interrupt handler will be called when a frame is received"
In this quote, does the term "interrupt handler" means the irq_handler.entry?

2, If that is true, does that mean each frame receive would trigger an
individual irq_handler(It seems this is non-NAPI condition)?

3, Besides NAPI, I also found Rnx and Tnx mode of network card, does
that mean the same thing?

4,  in the ref:https://web.archive.org/web/20111030030517/http://www.linuxfoundation.org/collaborate/workgroups/networking/networkoverview
it also says that:"The same interrupt handler will be called when
transmission of a frame is finished and under other conditions.
(depends on the NIC; sometimes, the interrupt handler will be called
when there is some error)."
Does that mean netdev.hard_transmit would trigger the same irq_handler
used in netdev.receive? But I can't profile any irq even soft irq, Is
that a "NIC depended" problem?

I am using virtio network device.

2015-12-11 1:35 GMT+08:00 William Cohen <wcohen@redhat.com>:
> On 12/09/2015 10:13 PM, Shane Wayne wrote:
>> Hi all:
>> I've recently doing some research on network and block device related
>> IRQ handling, so I use some database and network cache application as
>> the testbed.
>> I probed the following probes below:
>> netdev receive
>> netdev transmit
>> netdev hard_transmit
>> ioblock request
>> ioblock end
>>
>> irq_handler.entry
>> irq_handler,exit
>> softirq.entry
>> softirq.exit
>>
>> After the profiling, I notice that, when netdev receive is triggered.
>> It always appears within a softirq, and the softirq has a irq_handler
>> in front of it. It mean the sequence is like:
>>
>> irq_handler.entry
>> irq_handler.exit
>> softirq.entry
>>     one or several netdev receive
>> softirq.exit
>>
>> And I found that, the irq could happened in any process, even in my
>> application's context, but in most condition, it was found in swapper
>> process.
>>
>> And when it comes to netdev transmit, it seems that this action do not
>> relate with irq? The sequence just happens in the application context.
>>
>> netdev transmit begin
>>         netdev hard_transmit begin
>>         netdev hard_transmit end
>> netdev transmit end
>>
>> And it doesn't use irq?
>>
>> So here is my question:
>> 1, it seems that irq_handler from network device just be responsible
>> to notify the package arrival, and the actual handling was done by the
>> following soft irq?
>
> Yes, that is correct.  The irq handlers try to minimize the time they run.  The irq task mainly schedule some task to do the majority of the work in the softirq.
>
> http://www.makelinux.net/ldd3/chp-10-sect-4
>>
>> 2, the irq handling seems could happen in any process, and do not stop
>> current context, but when the server is not heavily loaded, the irq
>> prefer to happen in swapper process?
>
> irqs can happen any time they are not masked.  There are sections of kernel code that disable them to make sure that certain operation are atomic.
>
>>
>> 3, Or just because the swapper take more cpu time when the server was
>> not heavily loaded?
>>
>> 4, does the netdev send or ioblock request really do not need irq's assistant?
>
> The netdev and ioblock works needs to be scheduled.  The irq are indicating that the device needs feeding/attentions.  The alternative would be to do polling which would be probably be less efficient.  However, the NAPI netork code does going in to a polling mode to reduce the number of interrupts
> (https://en.wikipedia.org/wiki/New_API).
>
>>
>> 5, if this is true, how could we determine when the netdev transmit
>> finish it's job? is the time of netdev hard_transmit end correct?
>
> Actually it can be hard to determine precisely when the transmit has finished sending the packet.  The packet is given to the device, but it could take some time for the device to complete sending it on the wire.  One could look and when the packet buffer is freed for an upper limit on the time it took to send.
>
>
> -Will



-- 
------------------------------------------------------------
ZHANG Xiao

The Institute of Computer Software Engineering and Theory in Xi'an
Jiaotong University
Master-and-Doctoral Program Student

ACM Student Member
CCF Student Member
IEEE Computer Society Sister Society Associate Member

Xi'an Jiaotong University
Bachelor of Engineering in Computer Science and Technology

The Chinese University of Hong Kong
International Asia Study Program in CS/CE/EE/IE