So a couple weeks ago I explained something I was trying to do.  I
have a site that periodically pulls headlines off certain web sites,
then indexes them for display on my site.  It does this via two
scripts -- one which 'pulls' the html of the various sites & saves it,
and another which 'parses' these files to get the right links.

I wanted the scripts to execute every 15 minutes so that my site would
automatically update.  I received an answer on here regarding use of a
crontab.  I set up the crontab to execute a little unix script, which
switched into the cgi-bin directory and executed the two perl scripts
in sequence.  Everything seemed to work fine for about 2 weeks,
although I did notice that, sometimes, when I executed one of the perl
scripts manually, it would stall out and I'd have to exit with Ctrl-Z.
 I know it was an intense script with lots of file xferring, and I
suppose a non-responsive site could cause the script to crash.

Well, apparently, last nite, my little operation crashed my server
(oops!)  The sysop told me I had 100-150 instances of my little unix
scripts running simultaneously.  Apparently they were not always
exiting correctly -- I assume when the perl scripts stalled, the unix
scripts would just sit there perpetually open, waiting for them to
finish.

So my question is, how do I rectify this?  Is there a foolproof way of
making sure that all instances are killed even if they've somehow
stuck in an endless loop?  Is there a crontab command I could add that
would kill everything running once an hour or something?  In short,
how do I avoid getting myself banned from this server??

Thanks!!

First of all, in order to kill a running command, you must know what
the Process ID (PID) is.  The pidof command will give you the PIDs of
all running applications with a given name (try 'pidof httpd').  If
your only Perl scripts are ones you want killed hourly, you can use
pidof Perl to get a list of PIDs for all instances of Perl, yours and
other users'.  Then you can feed these numbers to kill, which will to
try to kill the processes.  Killing processes owned by others will not
work, but your running processes will be killed.

A more elegant approach would be to use ps ax to list all running
processes and to search (using grep and awk, using Perl, or some other
way) for only the processes you want to kill.  For example, if your
script is run as '/usr/bin/Perl foo' you could get the PIDs of only
those Perl processes, as opposed to *all* Perl processes, and kill
them instead.

Once you have written a simple script to do this, set it up to run
hourly using crontab just as you did with your other two scripts.

---

Request for Answer Clarification by tjsnod-ga on 05 Oct 2002 00:38 PDT

Hmmm...not sure what the limitations on my unix shell account are, but
when I type 'pidof' (or pidoff httpd), I get "command not found" msg. 
do you have any alternative ideas?

incidentally, my account is not that complicated -- one unix script
executing two perl scripts.  i assume when they stall, both the perl
and unix scripts after left in a perpetual loop.  but i'm happy
killing *everything* since i have limited access and assumably would
have no effect on other users.

thanks!

---

Request for Answer Clarification by tjsnod-ga on 05 Oct 2002 00:44 PDT

ok, i just figured out that the function 'jobs' can display process
ids w/ arg -l.  however, even if i open two instances of the shell
account, write a little 'c shell' script that pauses for 5 minutes,
execute it, then flip over to the other instance and type 'jobs' i
don't get anything listed.  is there a reason the 'jobs' command is
not reflecting the fact that there's a shell script sitting over there
running for 5 minutes?  these little c shells are precisely what was
crashing the system...

---

Clarification of Answer by haversian-ga on 05 Oct 2002 02:14 PDT

The pidof command works under the BASH shell (bourne again shell), and
many others, but may not work in all cases.

Can you tell me what version of unix (linux, solaris, AIX, BSD, etc.)
you are running on, and what shell you are using?  You mentioned the C
shell in your second clarification - was that used only for testing,
or are you using csh as your default shell?  Do you have access to
tcsh or bash (try 'which tcsh' or 'which bash' or just try running
these alternative shells and you will either get a 'command not found'
error or a new prompt.)  Also, to make this as short as possible,
could you provide the exact command being run by your crontab?  I do
not need a listing of the actual perl scripts, but if crond is running
a shell script which in turn runs perl, I would like a printout of the
shell script.  I assume it is only a  few lines long.

If you happen to be using Hurricane Electric, I can give you exact
commands, as I have access to their systems.

---

Request for Answer Clarification by tjsnod-ga on 05 Oct 2002 08:13 PDT

my server runs linux.  i seem to have access to the bash and tcsh
shells, as when i type them in they give me a directory (e.g., 'which
bash' returns '/bin/bash').

i don't have anything running at the moment, due to the earlier crash.
 when it was up and running, i had a csh script (choosing that shell
for no particular reason) that:

1) changed to the cgi-bin directory
2) executed the slower script (the one that sometimes seems to stall
out)
3) slept for 1 minute
4) executed the other perl script

The crontab was set to execute this csh shell script, named 'go',
every 15 mins.  i think the command was:

0,15,30,45 * * * * csh go


thanks for the clarifications!!

---

Clarification of Answer by haversian-ga on 05 Oct 2002 13:08 PDT

Ok.  You are unlikely to be able to use pidof, as it is almost
certainly restricted to the superuser, an oversight of mine.  ps on
the other hand should be accessible to you.

When I create a simple script which waits for 1 minute and then prints
Finished, it shows up in the process (ps) screen as follows:
28303 pts/0    S      0:00 /bin/sh ./wait.csh
28304 pts/0    S      0:00 sleep 60

The first line is the script itself, and the second line is the wait
command it has executed.  You will likely have two lines showing up as
well, one for your script, and one for perl.  If you kill the script,
you will stop it from executing more commands, but will have no effect
on the perl process which is running out of control.

If you have a shell script called foo which executes a perl script
called bar, you should kill the perl process running bar.

Type 'ps ax' and you should get a listing (much longer than this) but
with lines like

 1039 tty6     S      0:00 /sbin/mingetty tty6
 4888 ?        S      0:00 /usr/local/apache/bin/httpd
28205 ?        S      0:00 /usr/sbin/sshd
28206 pts/0    S      0:00 -bash

First, you need to strip off all lines not containing your script bar.
 To do this, use grep:
ps ax | grep bar

Next, you need to strip off everything following the first number. 
Here we have some difficulty because of my relatively weak grasp of
regular expressions.  You will need to make sure that the only things
you have in the second column (the one with ?, pts/0, and tty6 in my
example) start with either a t, a p, or a ?.  If there are more, you
will need to edit a file in a sec.  To do this stripping, I am using
sed.  The sed script I have is called sed.txt:
s/\ ?.*//
s/\ \t.*//
s/\ \p.*//
s/\ *//

To invoke it, run sed -f sed.txt.  Appending this to the code we
already have gives you 'ps ax |grep bar | sed -f sed.txt'

At this point, you may be able to proceed on your own.  Otherwise, I
have a somewhat inelegant, but functional solution.  Change the last
line of sed.txt to read
s/\ */kill\ /

Then touch a file called sedout.sh (touch sedout.sh)
Make it executable (chmod 755 sedout.sh)

Then run 
ps ax |grep bar | sed -f sed.txt > sedout.sh
which will capture the PIDs of all processes matching bar and output a
simple shell script to kill them all.  Next, to run the shell script,
type
./sedout.sh

Once you have this all set up and running properly (You will need to
replace bar with your particular command name, for example), create a
shell script which runs
cd your_favorite_directory
ps ax |grep bar | sed -f sed.txt > sedout.sh
./sedout.sh

and setup crontab to execute that script once an hour or however often
it needs to be run.

Hope that helps!  It works for me under bash and csh on RedHat 7.3,
but you may have subtle differences in your configuration such as a
different version of sed, which may need working around.

-Haversian

thanks for all the follow-up!