On 03/23/2012 04:19 PM, Josh Fisher wrote:
>
> On 3/23/2012 12:49 PM, Phil Stracchino wrote:
>> On 03/23/2012 11:58 AM, Josh Fisher wrote:
>>> A FIFO queue makes sense, but how will SD know when it has completed
>>> writing all of the spool files for a particular job?
>> The same way it knows it's finished writing all the data for a
>> particular job now, I assume.
>
> I don't think so. Each SD-FD connection is handled in its own thread. At
> the beginning of append_data(), begin_data_spool() is called to set the
> system device that will be written to. The thread then reads from the FD
> connection and writes to that device until all of the data has been sent
> by the FD. It then closes the FD connection and calls
> commit_data_spool() to read the spooled data and write it to volumes (or
> discard_data_spool() if there was an error.
>
> OK. That could be modified to repeatedly call begin_spool_data() and
> commit_data_spool() to split the spool file into N 1 GB files. But note
> that each job is in its own thread, so when it does the
> commit_data_spool(), the data is either written to a volume or there is
> an error. Either way, when commit returns, it knows the status. You are
> talking about a commit_data_spool() that doesn't write, but instead
> queues the file to be written by another thread. Consider the case where
> the data is less than 1 GB and when the commit_data_spool() queues the
> file to be written by the spool manager thread. Let's say it queues just
> fine. Then the append_data() will see that as a successful job and
> terminate, and the job will complete successfully. Meanwhile, say there
> is a problem with a volume and the actual write, being done by the spool
> manager thread, fails. Now we have a job that thinks it has succeeded,
> yet no data was written to tape.
>
> I think there has to be some sort of communication, and the thread doing
> the append_data() has got to wait and find out if the spool manager
> actually managed to write the data to tape, which is quite a bit
> different than the current way.
I see your point. Clearly there has to be communication of the whole
process, yes. It is only reasonable that each spool file would be
tagged with the JobId it belongs to, and if the "available for
despooling" flag were accompanied by a "final segment" flag, then upon
completing that final segment (or failing to completely write any
segment), the spool manager can send a message back to the job thread
saying either "Job successfully completed" or "Job failed".
--
Phil Stracchino, CDK#2 DoD#299792458 ICBM: 43.5607, -71.355
ala...@caerllewys.net ala...@metrocast.net p...@co.ordinate.org
Renaissance Man, Unix ronin, Perl hacker, SQL wrangler, Free Stater
It's not the years, it's the mileage.
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Bacula-devel mailing list
Bacula-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-devel
