Creating a daemon would work, of course, but I think it would be overkill. Like you said, you don't want to implement something that is too complex. Given the limitations of the file system you are using (I would think that Linux would allow an exclusive access lock. I'm not a Linux programmer yet.
) your process could look something like this (in pseudocode):
Code:
if Create lock file with O_CREAT | O_EXCL flags succeeds then
// At this point, we know we're good to go and have exclusive access to data file
Get list of files that match the queue files pattern e.g. Q*.UPD or whatever.
Sort the list according to the order that they occur
Read and apply the updates from the files and then delete them.
Process the update of the current process
Delete lock file
else
Get unique ID
Create update file with name of Q<unique id>.UPD
Write update request to file
Close update file
end if
I don't know how much information you have available to you in PERL, but if you can get the thread id or a high resolution timer value, use that for the unique id. A timer value would be helpful for maintaining the order of the updates.
The beauty of this process is that it will always complete without a wait. Since you process the updates in the order that they occur it will be seamless to the user. No lock is necessary on the UPD files since they are unique. In the specific case of User Rides, whenever someone looks up a ride the counter is incremented so this process will pretty much always occur.
Your autobackup strategy must consume some time. Removing the necessity for that should certainly help.
As far as the threads not consuming CPU time, you are correct. The load is occurring in the task manager. Each time a thread uses its time, the task manager has to determine if the next thread is ready to run. I'm assuming that you are using some sort SLEEP function as opposed to YIELD, CHECK TIME, YIELD process. A SLEEP function probably just suspends a thread and sets a counter that decrements each time the task manager is called until it goes to zero.
Also, don't forget the initial scenario I mentioned. Since you are waiting for 1 sec, it's possible for one thread to give up a lock and another to take it before the waiting thread completes its wait. When it finally gets around to checking again, it will see that it is still locked and wait again. As load increases (the number of threads), this race condition is more likely to occur. If timed right, some threads could be starved for access for many seconds while only a few get to actually do anything. This is why your thread count goes so high.
These types of synchonization problems will still occur with Cold Fusion and SQL server, just in different areas.
[This message has been edited by jimberg (edited 09-13-2000).]