I don't think I even mentioned Windows stuff in my last message.
Most of my major multithreaded programming was with kernels and schedulers that I wrote. When I did programs for IIS that were fairly similar to User Rides and pounded it with 120 simultaneous users, the highest thread count I ever got was 7 threads. If everything positive about Linux that I have read is true, you should be able to get your max count to at most half of that.
Regardless of how efficient the scheduler is in the OS, however, having tasks vying for resources by having them wait for an arbitrary, predetermined period of time is still a major problem because of the issue creating a race condition.
I think you pretty much have what I am saying, except for the part about waiting only if queue files exist. The process will be extended a little, but would still be continuous. The program will never wait.
As far as getting rid of the locking file, I went to PERL.ORG to see if I could find some info on file locking. Does the
FLOCK <file handle>, 2 not work with your implementation of PERL? That would give you exclusive access to write to the file without needing to create a lock file. In the FAQ, their example was a page hit counter, so I'm pretty sure that is what I was looking for. If the FLOCK fails, run the ELSE code in the above pseudo code.
If you use SQL Server 7, yes, it will be much easier.