How can dirty buffers be flushed to disk (similar to invoking sync)
from a bottom half tasklet?  According to Alessandro Rubini's Linux
Device Drivers, "bottom halves cannot sleep, cannot access user space,
and cannot invoke the scheduler".  My interrupt handler works fine
with "safe" functions but will not insert with with a sync system
call.  There must be a way to call a function that will flush the
dirty buffers.  My goal is to use an interrupt handler to flush dirty
buffers.

---

Request for Question Clarification by maniac-ga on 22 Jan 2003 19:59 PST

Hello Madronna,

Are you looking for a way to initiate a "sync" (to flush dirty buffers
to disk) from the taslket? The normal definition of sync is to...
 - wait for all pending writes to complete
 - queue all dirty buffers to be written
and it is the first step that your tasklet cannot perform. If the
second part was initiated by your function - would that be "good
enough" (and then perhaps some way to find out the writes were
completed)  or something else?

  --Maniac

---

Clarification of Question by madronna-ga on 23 Jan 2003 09:26 PST

Thank you for your question.  If by queuing the dirty buffers the
flush is initiated from the tasklet (activating the bdflush or kupdate
kernel thread, for example), and then confirming the writes were
completed would be good enough.  My goal is to kick off a buffer flush
asap from the tasklet.

---

Request for Question Clarification by maniac-ga on 29 Jan 2003 16:47 PST

Hello Madronna,

Based on my analysis (see comment) it does not seem to be possible to
directly flush the buffers to disk from an interrupt. Would you accept
an answer that uses a "helper task" to initiate the disk flush?
  --Maniac

---

Clarification of Question by madronna-ga on 30 Jan 2003 17:32 PST

I would accept "helper task" if it were a bit more specific.  Are you
suggesting the use of signals to communicate with a user space
process, or some other form of communication?  In the case of signals,
is a call to flush the buffers signal handler safe?

Hello Madronna,

The concept of a helper task (or process) is very simple and there are
some good source code examples in the kernel tree. I will outline a
few ways to implement this that should meet your needs.

If you run ps or top on your Linux system, you should see a number of
kernel processes listed including "kswapd," "bdflush", and others. The
swap daemon (kswapd) is declared in
  mm/vmscan.c (look near the end of the source file)
and is started with a kernel_thread call. Bdflush is defined in
  fs/buffer.c (again, near the end of the file)
and is started in a similar manner. There is actually another example
described in the Linux Device Drivers 2nd Edition book - look at
module loading (pages 305-311) for an explanation of creating a kernel
thread [it describes modprobe]. Also on page 311 - it describes
call_usermodehelper which indicates you can run as a child of keventd.

Note that for these two examples, the source code for them is in the
kernel, yet they operate basically as user mode processes (there *are*
a few exceptions). As such, they can do pretty much any system call
they need to "get the job done". Both of these for example, initiate
disk I/O, kswapd to the swap partition and bdflushd to the file
system.

The method used to wake these processes up is basically a semaphore.
It is the process is basically in an infinite loop, waiting on that
semaphore. When it resumes, it does the task assigned (in the case of
bdflush, go through the buffer queues, do the I/O, move the completed
work into other queus), and goes back to sleep - waiting for the next
wake up.

In your case, I see a few techniques to create the helper task
 1 - start the helper task as part of your module initialization (like
kswapd or call_usermodehelper)
 2 - install an init script as part of your driver install; the script
simply runs the helper as a background task after the driver is loaded
I can see a few others, but after #1 (if it works) or #2 (use if #1
doesn't), they don't seem to palatable to me (e.g., require the
application to do some of the work...).

The waiting condition can be done as
 1 - a semaphore
 2 - a blocking I/O operation (e.g., "recv")
 3 - a signal
or some other synchronization mechanism. Best is to declare the
semaphore in the driver and make it visible to the helper task
(directly if in the driver source code). I would do #2 only if you
could not get #1 to work. #3 is a reasonable alternative; a driver we
have uses signals (on a clock tick, it can wake up more than one
task), but I prefer using blocking I/O calls because it tends to be a
simpler solution. I can dig up the specific calls for each if you need
them.

If the helper task is not part of the driver, be sure to
 - lock it into memory - use mlockall(MCL_CURRENT|MCL_FUTURE)
 - give it real time priority - use sched_setscheduler - set
SCHED_FIFO for real time priorities
You may have to set the task priority if you start it as a
kernel_thread - I can't tell for sure from the source nor the
documentation.

The latency I mentioned (a millisecond or so) is basically the time to
 - return from the bottom half (to the scheduler)
 - schedule the "highest priority process" (usually your helper task)
 - do the sync call from the helper
You might get preempted by another interrupt, but that should not be
too much of a problem either for the kind of task you are performing.

If some of this is still to vague, let me know so I can expand on the
answer. Good luck with your work.

  --Maniac

Hello Madronna,

I took a look at the source code in 
  /usr/src/linux/fs/buffer.c
and I believe the answer to your question is "it can't be done".
Because of that, I'll post this as a comment and suggest some
alternatives. If you want to accept as an answer - make another
clarification so I can repost it. This also allows another researcher
to suggest a more suitable answer if they can think of one.

For background, buffer.c includes functions such as

  sys_sync - what gets called by the sync (2) system call. It
basically will call sync_buffers, requesting a wait.
  sync_buffers - takes two parameters, the device and a "wait flag".
If the wait flag is zero, it claims to not wait. Looking inside, it
first calls write_unlocked_buffer and if wait is non-zero, does a few
more steps.
  write_unlocked_buffer - gets the LRU buffer lock & then calls
write_some_buffers
  write_some_buffers - is a little complicated, but the "buffer write"
appears to be set up by a function called __refile_buffer which simply
moves an item from one LRU list to another (hence the need for the LRU
buffer lock). I can't see where it actually schedules the I/O to take
place - I think that is somewhere else. I also see here a conditional
schedule which would rule out calling this from the bottom half
tasklet.

I was hoping to find one of these functions suitable to call from the
tasklet. What I found is that it looks doubtful that your driver has
visibility into the functions that might do the job w/o the danger of
waiting. The conditional schedule deep in write_some_buffers was
particularly disturbing to find.

I guess I have a question for you. Why is a "helper task" not suitable
for this job? If you don't mind a millisecond (or so) delay, you could
to the following...
  - create a task as part of the driver initialization (or as part of
your application)
  - use one of the driver interfaces to wake that task up when buffers
need to be flushed
  - the helper task, running in user mode could then do the sync
(fsync?) system calls
Run it at "real time priority" and locked into memory if timing is an
issue.

The other alternative I can think of to get an answer is to ask
politely on linux-kernel and see what they say. I tend to hang out on
linux-mm instead (can get serious technical questions answered, far
fewer messages per day) and my experience is someone will be glad to
help you. You could also send a question to
  mailto:bookquestions@oreilly.com
as noted in the preface to Linux Device Drivers.

Good luck.

  --Maniac