File IO is one of more error prone activities with low understanding by general software dev community.
It is actually rare to see a piece of code that is NOT broken.
I had once observed a piece of software write a million files, all empty, because it could not handle situation where it could create file but not write it.
It is quite normal for files to be truncated because written directly to destination file rather than through temp. Then script gets killed and next instance assumes file exists so it resumes from next.
Most software can do the job if everything works, but almost none can correctly handle every error condition.
File IO is one of more error prone activities with low understanding by general software dev community.
It is actually rare to see a piece of code that is NOT broken.
I had once observed a piece of software write a million files, all empty, because it could not handle situation where it could create file but not write it.
It is quite normal for files to be truncated because written directly to destination file rather than through temp. Then script gets killed and next instance assumes file exists so it resumes from next.
Most software can do the job if everything works, but almost none can correctly handle every error condition.