here is a maillist discussing the problem about why cannot sleep in the ISR from linux newbie. it is worth reading throughout if you do not clear about this question.
~~~~~~~~~~~~~~~~~
发件人： Learning Linux 发送时间： 2007-05-14 14:40:54 收件人： kernelnewbies@nl.linux.org; linux-newbie@vger.kernel.org 抄送： linux-kernel@vger.kernel.org 主题： Why can't we sleep in an ISR?
I have a very basic doubt here ... what makes it impossible to sleep in an ISR? I mean, I know that the kernel preemption is disabled and the kernel will panic, but I could not understand why?
TIA,
==================== Learning Linux wrote: > I have a very basic doubt here ... what makes it impossible to sleep > in an ISR? I mean, I know that the kernel preemption is disabled and > the kernel will panic, but I could not understand why?
The impossibility to sleep comes as a natural consequence of the fact that ISRs have to be fast.
An ISR has to be fast in order to allow other system tasks to happen. Otherwise it will do a lot of work (like busy waiting) and any other interrupts will fail to get handled. That would completely render the system unusable.
On the other hand, when you sleep in a given context, the scheduler is invoked. That means that the ISR routine would be replaced by a process from the scheduler's queue(s). But how would you ever know to reschedule the ISR? After all, it is asynchronous.
This is why ISRs (top halves) and tasklets/softirqs (bottom halves) are said to be running in interrupt context; processes are said to be running in process context.
Interrupt context has very strict rules like being unable to sleep or do a large amount of work (this will render the system unable and, quite likely, a panic would occur).
Hope this helps. You should be read Robert Love's excellent "Linux Kernel Development 2nd Edition". It is quite readable and has a pleasant way of showing you the basic kernel concepts. Chapter 6 is concerned with interrupts and interrupt handling routines (ISRs).
Razvan
============================
Because the interrupt which you are serving in the ISR has been masked to avoid preemption( true for maskable inetrrupts ).Any locks you are holding in ISR are now with you solely. So, if you try to sleep you take the locks you untill you are rescheduled and complete. This may lead to a deadlock for the lock resource. Thus to avoid this, kernel developers disable preemption and also on every schedule check if you are *in_interrupt*, if you are, you are probably doing something really bad.So its better to panic in such case.
This is my naive understanding. Please CMIIW.
========================
> Because the interrupt which you are serving in the > ISR has been masked to avoid preemption( > true for maskable inetrrupts ).Any locks you are holding in ISR are now with > you solely. So, if you try to sleep you take the locks you untill you are > rescheduled and complete. This may lead to > a deadlock for the lock resource.
Ok, but how about an ISR, that does not take any locks? Why can't we sleep in SUCH an ISR?
AFAIK, taking a lock disables kernel preemption, and hence it is not allowed to sleep. So I think my question would boil down to why is sleeping not allowed when the kernel preemption is disabled.
LL
========================================
> > On the other hand, when you sleep in a given context, the scheduler is > invoked. That means that the ISR routine would be replaced by a process > from the scheduler's queue(s). But how would you ever know to reschedule > the ISR? After all, it is asynchronous.
Uh, this is what I have my doubt about. Yes, the scheduler may try to replace the current process (already interrupted by the ISR) with another process.
So, what is the problem with that? Could you please elaborate on the part "how would you ever know to reschedule the ISR? After all, it is asynchronous"?
====================================== >> >> On the other hand, when you sleep in a given context, the scheduler is >> invoked. That means that the ISR routine would be replaced by a process >> from the scheduler's queue(s). But how would you ever know to reschedule >> the ISR? After all, it is asynchronous. > > Uh, this is what I have my doubt about. Yes, the scheduler may try to > replace the current process (already interrupted by the ISR) with > another process. > > So, what is the problem with that? Could you please elaborate on the > part "how would you ever know to reschedule the ISR? After all, it is > asynchronous"?
Well, the ISR code has to be resumed at some point doesn't it? How would you know how to "get back" at executing it? There's no ISR scheduling queue, as there is no process. An ISR can interrupt any given process (that's what I meant by asynchronous).
And even if that weren't a problem you would still have to deal with another problem: if an important interrupt (like the timer interrupt) would sleep and would (somehow) manage to get rescheduled, you would lose any notion of time within that system; it will inconsistent, and it will break all other subsystems (like scheduling) (the process time slice would lose its meaning)
Razvan
=========================== > > I have a very basic doubt here ... what makes it impossible to sleep > > in an ISR? I mean, I know that the kernel preemption is disabled and > > the kernel will panic, but I could not understand why? > > Because the interrupt which you are serving in the > ISR has been masked to avoid preemption( > true for maskable inetrrupts ).Any locks you are holding in ISR are now with > you solely. So, if you try to sleep you take the locks you untill you are > rescheduled and complete. This may lead to > a deadlock for the lock resource.
Ok, but how about an ISR, that does not take any locks? Why can't we sleep in SUCH an ISR?
Adding extra complexity is to be refrained from, because such ISRs are very few IMHO. Moreover on a SMP system why wouldn't you need a lock? preemption is disabled only on the local processor i guess. CMIIW
AFAIK, taking a lock disables kernel preemption, and hence it is not allowed to sleep. So I think my question would boil down to why is sleeping not allowed when the kernel preemption is disabled.
Who will preempt you, in this case? i.e to sleep? No one, you will complete and therefore you should be really fast in doing it. Becasue you have just disabled kernel preemption. In other words ISR is not like any other process waiting to run, it is a special kernel control path which is special because of the reasons you have given. There is no task struct for your interrupt, no accounting as such AFAIR and etc etc.
It runs in current process's context, isnt it? HTH
~psr =====================================
> Well, the ISR code has to be resumed at some point doesn't it? How would > you know how to "get back" at executing it? There's no ISR scheduling > queue, as there is no process. An ISR can interrupt any given process > (that's what I meant by asynchronous).
Yes. The ISR can interrupt any process. But the scheduler is totally unaware of it (CMIIW). As far as scheduler is concerned, it thinks that the processor is still executing the code of the process that ISR interrupted. In fact, all the time ISR executes is deducted from the timeslice of the process that was interrupted.
Assuming the simple case of 8KB stacks (ISRs share the process stack). So the scheduler can still perform a context switch saving and restoring the save of the process it THINKS is currently executing (The PC will actually point to an address in ISR) and later on restore from where it left?
============================== > > > AFAIK, taking a lock disables kernel preemption, and hence it is not > > allowed to sleep. So I think my question would boil down to why is > > sleeping not allowed when the kernel preemption is disabled. > > Who will preempt you, in this case? i.e to sleep? No one, > you will complete and therefore you should > be really fast in doing it.
Ok, lets forget about ISRs and interrupts. Let us say I have a kernel thread running in process context, and it takes a lock (thus disables preemption) which is used ONLY by this thread. Now it WANTS to sleep for some time, while holding a lock.
There surely ARE a lot of runnable processes in the system, who can run. Now technically sleeping, what stops this process from sleeping? Why can't this process sleep safely?
Uh, just to clarify, I totally understand that this is not allowed and will be a bad design. But I just want to understand that in terms of kernel code, what STOPS this process from going to sleep?
=============================
> >> On the other hand, when you sleep in a given context, the scheduler is > >> invoked. That means that the ISR routine would be replaced by a process > >> from the scheduler's queue(s). But how would you ever know to reschedule > >> the ISR? After all, it is asynchronous. > > > > Uh, this is what I have my doubt about. Yes, the scheduler may try to > > replace the current process (already interrupted by the ISR) with > > another process. > > > > So, what is the problem with that? Could you please elaborate on the > > part "how would you ever know to reschedule the ISR? After all, it is > > asynchronous"? > > Well, the ISR code has to be resumed at some point doesn't it? How would > you know how to "get back" at executing it? There's no ISR scheduling > queue, as there is no process. An ISR can interrupt any given process > (that's what I meant by asynchronous).
Yes. The ISR can interrupt any process. But the scheduler is totally unaware of it (CMIIW). As far as scheduler is concerned, it thinks that the processor is still executing the code of the process that ISR interrupted. In fact, all the time ISR executes is deducted from the timeslice of the process that was interrupted.
=========================== Learning Linux wrote: > I have a very basic doubt here ... what makes it impossible to sleep > in an ISR? I mean, I know that the kernel preemption is disabled and > the kernel will panic, but I could not understand why? First: an ISR is meant to be very quick. It is supposed to do only a minimum of work needed to service the interrupt, then exit. This is important, as other interrupts might be blocked during your ISR. Sleeping is out of question, even a long-running loop in no-no.
Second: You don't ever need to sleep in an ISR anyway. Complicated work that might take time or might need to sleep is not supposed to be in an ISR. If you think you have a need, tell us what you're up to and hopefully someone will explain how do do things properly.
When an interrupt happens that needs complicated servicing, the ISR don't do the whole job. It just acknowledges the interrupt, perhaps does a few things with the device in question, then it exits. It leaves the rest of the work for a bottom half or kernel thread or something like that. Kernel threads may sleep . . .
Helge Hafting
================================= My understanding is as follows.
Whenever the kernel code sleeps, it means the latest process running in user space will have to wait for the event on which the kernel code sleeps.
It makes sense for an exception handler to sleep because an exception handler always serves the latest process running in user space. So a process can complain nothing for it having to wait for the event on which the exception handler in its own context sleeps.
It makes no sense for an ISR to sleep because an ISR does not necessarily serve the latest process (that is, the interrupted process). It makes no sense having a process wait for the event having nothing to do with it.
I could be wrong, so please correct me if the understanding is not right.
Dong Feng
=======================
On 5/14/07, Learning Linux wrote: > Ok, but how about an ISR, that does not take any locks? Why can't we > sleep in SUCH an ISR? > LL > -
The killer reason why you can't sleep in an interrupt is because an interrupt is not associated with any context in the first place. What is a context, then? It is the state information for a process. This includes the kernel and userspace stack pointers, the register set, and the page tables for that process. The scheduler has access to all this information, to preempt one process and run another. Contrary to this, an interrupt, depending on the version of your kernel and arch, uses a separate irq stack or the kernel stack of the interrupted process. An irq is not a context but merely a temporary execution to be concluded asap.
Hope this helps, Bahadir
======================================= I agree that the reason an interrupt can not sleep is because an interrupt is not associated with any context. But I do not agree that it is specifically because the scheduler can not *resume* the context.
In early version, the ISR always borrow the stack of the currently running process, so if the kernel design had allowed ISR sleep, it would automatically be able to implement the context switch naturally. But ISR sleep has been forbidden from the very beginning.
The reason why kernel design does not allow ISR sleep is because a process should not be forced to wait for a event that irrelative to itself. When an exception handler sleep, the event it sleeps on is always in some sense related to the process incurring that exception. But if an ISR was allowed to sleep, the event it sleeps on would have nothing to do with the process being interrupted.
So my understanding is, the forbidden of ISR sleep is not because of the difficulty to resume context, but because a process should not wait for irrelative event.
Dong Feng ================================
On 5/14/07, Bahadir Balban wrote: On 5/14/07, Learning Linux wrote: > Ok, but how about an ISR, that does not take any locks? Why can't we > sleep in SUCH an ISR? > LL > -
The killer reason why you can't sleep in an interrupt is because an interrupt is not associated with any context in the first place.
good enough, but i have a query regarding this then. On a 8K kernel stack system, doesn't interrupts share the stack associated with the current process which was interrupted? Doesn't interrupt steals the CPU slice time allocated to the running process to run? Doesn't it run in current process's context ?
What am i missing here?
Thanks ~psr ===================================== > > good enough, but i have a query regarding this then. > On a 8K kernel stack system, doesn't interrupts share the stack associated > with the current process which was interrupted?
Yes, I think so.
> Doesn't interrupt steals the CPU slice time allocated to the running process > to run?
I don't think so but I am not sure.
> Doesn't it run in current process's context ? >
No. I think the concept of process context is a higher-level logical concept. Though the interrupt share stack with the interrupted process, in my opinion it logically does not share the context with the process.
> What am i missing here? > > Thanks > ~psr >
But I do not see the exact relationship between your specific queries and the original question. Could you elaborate?
Dong Feng
===================================
On 5/15/07, Dong Feng wrote: > > good enough, but i have a query regarding this then. > On a 8K kernel stack system, doesn't interrupts share the stack associated > with the current process which was interrupted?
Yes, I think so.
Yes it does. > Doesn't interrupt steals the CPU slice time allocated to the running process > to run?
I don't think so but I am not sure.
Aliter, i think so.How can an interrupt's execution time go unaccounted then? I guess it does not, only the current processes running time is accounted for. Thoughts?
> Doesn't it run in current process's context ? >
No. I think the concept of process context is a higher-level logical concept. Though the interrupt share stack with the interrupted process, in my opinion it logically does not share the context with the process.
Yes, you are right as i can infer. thats why ISRs are special kernel control paths. But the poster asked, why can't we make ISRs to share context with the interrupted process if it not holding any locks? This is rather a desing issues IMO rather than imlementation, isnt it?
I guess even if it is possible, it would over complicate the handler code. Better trying to keep it simple i guess. Please CMIIW
[snip]
But I do not see the exact relationship between your specific queries and the original question. Could you elaborate?
My query here was related to your previous reply.Just wanted to pit some doubts as raised by the orirignal poster.That's it.
Thank you ~psr
================================
> > > > I don't think so but I am not sure. > > Aliter, i think so.How can an interrupt's execution time go > unaccounted then? > I guess it does not, only the current processes running > time is accounted for. > Thoughts? >
The interrupt handler's execution time will definitely defer the execution of the process, but I think it does not steal the process's time slice (the time_slice field not subtracted).
> > > Doesn't it run in current process's context ? > > > > > > > No. I think the concept of process context is a higher-level logical > > concept. Though the interrupt share stack with the interrupted > > process, in my opinion it logically does not share the context with > > the process. > > Yes, you are right as i can infer. thats why ISRs > are special kernel control paths. > But the poster asked, why can't we make ISRs to > share context with the interrupted process > if > it not holding any locks? This is rather a desing issues > IMO rather than imlementation, isnt it? > > I guess even if it is possible, it would over complicate the handler code. > Better trying to keep it simple i guess. Please CMIIW
My understanding is, the ISR is in different context from the process because of the definition of term *context*. In my opinion, to say two code pieces running in the same context means that two pieces of code has some logical relationship to meet a common objective. That's why I said *context* is a higher-level logical concept. It's not a concept defined in the level of hardware or code implementation, but instead in the level of logical. I think, by its definition, it makes no sense to say an ISR share context with the process interrupted by it because an ISR just randomly interrupts a process, with no logical relationship.
Dong Feng ======================== On 5/15/07, Dong Feng wrote: > > > > I don't think so but I am not sure. > > Aliter, i think so.How can an interrupt's execution time go > unaccounted then? > I guess it does not, only the current processes running > time is accounted for. > Thoughts? >
The interrupt handler's execution time will definitely defer the execution of the process, but I think it does not steal the process's time slice (the time_slice field not subtracted).
Ok, if i accept this, how does that explains this-
Following code snippet is from main schedule() function[kernel 2.6.20.1] -
Now, AFAIK runtime = now - prev->timestamp; will get you a value which ( prev->timestamp is *stamped* during previous switching of the tasks) is difference of this previous stamp and now value. Assuming before this schedule() an interrupt was served, then where did the time spent in ISR accounting is adjusted? I guess this is the code, where it is checking for a *possible* runtime to overshoot its stipulated NS_MAX_SLEEP_AVG and resetting this. Isn't it? And exactly why would that happen? Perhaps because the process was about to expire its slice when somebody barged in and stole its share and also overshot the stipulated time(may be very less though).
Can you help me in clearing this?
> > > Doesn't it run in current process's context ? > > > > > > > No. I think the concept of process context is a higher-level logical > > concept. Though the interrupt share stack with the interrupted > > process, in my opinion it logically does not share the context with > > the process. > > Yes, you are right as i can infer. thats why ISRs > are special kernel control paths. > But the poster asked, why can't we make ISRs to > share context with the interrupted process > if > it not holding any locks? This is rather a desing issues > IMO rather than imlementation, isnt it? > > I guess even if it is possible, it would over complicate the handler code. > Better trying to keep it simple i guess. Please CMIIW
My understanding is, the ISR is in different context from the process because of the definition of term *context*. In my opinion, to say two code pieces running in the same context means that two pieces of code has some logical relationship to meet a common objective. That's why I said *context* is a higher-level logical concept. It's not a concept defined in the level of hardware or code implementation, but instead in the level of logical. I think, by its definition, it makes no sense to say an ISR share context with the process interrupted by it because an ISR just randomly interrupts a process, with no logical relationship.
Right agreed.
Thanks ~psr
====================
> The interrupt handler's execution time will definitely defer the > execution of the process, but I think it does not steal the process's > time slice (the time_slice field not subtracted).
It will definitely be substracted from the process's time slice. Because the timeslice is substracted in timer interrupt, and does not differenciate if the process is executing ISR or not.
==================== Yes, you are right in this regard. An interrupt handler does steal the time slice from the interrupted process.
So now I think it is considered an acceptable deviation in calculating the process run time as well as determine process scheduling because an ISR should take very short time to return, in part as a consequence of the rule that ISR should not sleep.
Dong Feng ====================================
On Tue, 15 May 2007 pradeep singh wrote : >On 5/14/07, Bahadir Balban wrote: >> >>On 5/14/07, Learning Linux wrote: >> > Ok, but how about an ISR, that does not take any locks? Why can't we >> > sleep in SUCH an ISR? >> > LL >> > - >> >>The killer reason why you can't sleep in an interrupt is because an >>interrupt is not associated with any context in the first place. > > >good enough, but i have a query regarding this then. >On a 8K kernel stack system, doesn't interrupts share the stack associated >with the current process which was interrupted? Yes, you are right. >Doesn't interrupt steals the CPU slice time allocated to the running process >to run? Yes >Doesn't it run in current process's context ? You got it worng here. It runs on behalf of the process, but in the current process's context. For interrupts we have a dirrerent context altogether. > >What am i missing here? There are two different contexts of execution Process context Interrupt context > >Thanks >~psr > Thanks, -Rohit =====================================
On 15 May 2007 09:34:23 -0000, rohit hooda wrote:
On Tue, 15 May 2007 pradeep singh wrote : >On 5/14/07, Bahadir Balban < bahadir.balban@gmail.com> wrote: >> >>On 5/14/07, Learning Linux wrote: >> > Ok, but how about an ISR, that does not take any locks? Why can't we >> > sleep in SUCH an ISR? >> > LL >> > - >> >>The killer reason why you can't sleep in an interrupt is because an >>interrupt is not associated with any context in the first place. > > >good enough, but i have a query regarding this then. >On a 8K kernel stack system, doesn't interrupts share the stack associated >with the current process which was interrupted? Yes, you are right. >Doesn't interrupt steals the CPU slice time allocated to the running process >to run? Yes >Doesn't it run in current process's context ? You got it worng here. It runs on behalf of the process, but in the current process's context. For interrupts we have a dirrerent context altogether.
Ok, that explains a bit. how will you define an interrupt context?
Thanks ~psr
========================= Dong Feng wrote: >> Doesn't it run in current process's context ? >> > > No. I think the concept of process context is a higher-level logical > concept. Though the interrupt share stack with the interrupted > process, in my opinion it logically does not share the context with > the process.
No, the term context here has a specific meaning. It refers to those things which flow from the current pointer, including the virtual memory space, file descriptor table, current uid, and so forth. Because the current pointer is not changed on entry to an ISR, the ISR is executing in the context of the interrupted process, and thus uses that process' virtual memory, etc.
===================================
2007/5/16, Phillip Susi : > Dong Feng wrote: > >> Doesn't it run in current process's context ? > >> > > > > No. I think the concept of process context is a higher-level logical > > concept. Though the interrupt share stack with the interrupted > > process, in my opinion it logically does not share the context with > > the process. > > No, the term context here has a specific meaning. It refers to those > things which flow from the current pointer, including the virtual memory > space, file descriptor table, current uid, and so forth. Because the > current pointer is not changed on entry to an ISR, the ISR is executing > in the context of the interrupted process, and thus uses that process' > virtual memory, etc. >
If what you say were true, then an ISR would be running in the same context as the interrupted process. But please check any article or book, it will say ISR running in different context from any process. So ISR is considered in its own context, although it shares a lot of things with the interrupted process. I would only say *context* is a higher-level logical concept.
Dong Feng
==========================
Dong Feng wrote: > If what you say were true, then an ISR would be running in the same > context as the interrupted process.
Yes, and it is, as others have said in this thread, which is a good reason why ISRs can't sleep.
> But please check any article or > book, it will say ISR running in different context from any process. > So ISR is considered in its own context, although it shares a lot of > things with the interrupted process. I would only say *context* is a > higher-level logical concept.
Depends on which book or article you are reading I suppose. The generally accepted and often used thought is that ISRs technically are running in the context of the interrupted process, but because that context is unknown and therefore should not be used, it is often said that they run in no context, or outside of any context. Sometimes people then assume that because they run outside of any ( particular ) process context, they must be in their own context, but this is a mistake.
Phillip Susi
==================================
====================== Hi ... > I have a very basic doubt here ... what makes it impossible to sleep > in an ISR? I mean, I know that the kernel preemption is disabled and > the kernel will panic, but I could not understand why? Sorry, I reply back to the top "level". I try to read related chapters on Understanding The Linux kernel 3rd edition. I found something (page 144, Chapter 4 "Interrupts and Exceptions")...I quote them here: The price to pay allowing nested kernel control paths is that an interrupt handler must never block, that is, no process switch can take place until an interrupt handler is running. In fact, all the data needed to resume a nested kernel control path is stored in the Kernel Mode stack, which is tightly bound to the current process.
I hope it can shed a light for you regarding your confusion.
regards,
Mulyadi
==================================== OK. I think the gap between you and me is the definition of term *context*. If you go to Linux Kernel Development, 2nd Edition (ISBN 0-672-32720-1), Page 6, then you will read the following:
.... in Linux, ... each processor is doing one of three things at any given moment:
1. In kernel-space, in process context, ... 2. In kernel-space, in interrupt context, not associated with a process, ... 3. In user-space ...
This list is inclusive. ...
Maybe you prefer other terminology system, but I do like the above definition given by Robert Love. So maybe in your system *context* mean something at hardware level and you say ISR is in process context, but I think it is more like a logical level and agree with Rovert's definition.
And in hardware level, Robert's *context* definition also mean something specific, that I started to be aware of. That is, *in the same context* means a kernel-code is triggered by a user-space code. *in different context* means a kernel-code is triggered by an external interrupt source other than a user-space code.
Context has nothing to do with whether an ISR borrow any data structure of a process, instead, its something logical or related to causality. Dong Feng
