Hi Joe,
As an initial disclaimer, the BulkMutateJobService is not designed to
increase the speed of operations. It's main goal is to reduce the
overhead on the client side when making large numbers of operations
and to have the backend handle certain types of logic automatically.
Given the small number of the operations you are processing (~3000) it
probably makes sense to use the synchronous services.
As for the slowness seen with the BMJS, something that jumps out in
your code is how you are creating your streams, parts, and job. The
createAndBeginBulkJob() method takes as a parameter operationsByPart,
which is an array of arrays of operations. This allows you to pass in
multiple sets of operations which will be processed in different
parts. From what I can tell, you are instead passing in a single
array of operations. This side effect here I believe is that you are
creating a separate operation stream and part for every operation in
your array, which is creating a lot of unnecessary overhead.
Try the following change and see if that improves performance:
//Start Jobs
$jobId = $this->createAndBeginBulkJob($scopingEntityId,
array($operations));
Best,
- Eric Koleda, AdWords API Team
On Feb 8, 4:43 pm, Joe_Rocket <j...@netblue.us> wrote:
> OK, I've manage to migrate our application from v13 to v2009. We've
> got some serious speed issues in keyword generation with the Bulk
> Processing. So I'm wondering if someone could take a look and tell me
> if I could construct the Bulk jobs more efficiently. Thanks.
>
> Test Campaign Run
> -----------------------
> single campaign
> 15 ad groups
> 3 ads per adgroup
> 65 keywords per adgroup
>
> Test Results
> ----------------------
> v13 Run Time = 265.396441936s = 4.41 minutes
>
> v2009 Run Time = 7939.71670389 = 132 minutes!!!
>
> Our application code for v13 keyword function was was pretty much just
> wrapping Apility like this:
>
> public function createMultipleKeywords($keywordArray){
>
> $criterionObjects = addKeywordCriterionList($keywordArray);
>
> return $criterionObjects;
> }
>
> We'd batch create all keywords from one ad group at a time, which on
> average was about 50 keywords at a time. Any keyword failures causing
> the group to fail, our system would go back and reprocess that group
> one at a time.
>
> Now in converting to v2009, our new keywords function based off the
> Google Example code looks like this:
>
> public function createMultipleKeywords($keywordArray, $campaignId){
>
> $result = null;
> $operations = array();
>
> $adGroupCriteriaService = $this->adWordsUser-
>
> >GetAdGroupCriterionService('v200909');
>
> $count = 0;
> if(!empty($keywordArray)){
> foreach($keywordArray as $key=>$keywordRecord){
>
> // Create keyword.
> $keyword[$count] = new Keyword();
> $keyword[$count]->text =
> $keywordRecord['text'];
> $keyword[$count]->matchType =
> $keywordRecord['type'];
>
> // Create ad group bid.
> $bids[$count] = new
> ManualCPCAdGroupCriterionBids();
> $keyMaxCpc = $keywordRecord['maxCpc'] *
> self::GOOGLE_MICROS_PER_CURRENCY_UNIT;
> $bids[$count]->maxCpc = new Bid(new
> Money($keyMaxCpc));
>
> // Create biddable ad group criterion.
> $keywordAdGroupCriterion[$count] = new
> BiddableAdGroupCriterion();
> $keywordAdGroupCriterion[$count]->adGroupId =
> (float)
> $keywordRecord['belongsToAdGroupId'];
>
> $keywordAdGroupCriterion[$count]->criterion =
> new Keyword();
>
> $keywordAdGroupCriterion[$count]->criterion->text =
> $keywordRecord['text'];
>
> $keywordAdGroupCriterion[$count]->criterion->matchType =
> $keywordRecord['type'];
>
>
> $keywordAdGroupCriterion[$count]->destinationUrl =
> $keywordRecord['destinationUrl'];
> $keywordAdGroupCriterion[$count]->bids =
> $bids[$count];
>
> // Create operations.
> $operations[$count] = new
> AdGroupCriterionOperation();
> $operations[$count]->operand =
> $keywordAdGroupCriterion[$count];
> $operations[$count]->operator = 'ADD';
>
> $count++;
> }
>
> $bulkMutateJobService =
> $this->adWordsUser->GetBulkMutateJobService();
>
> $scopingEntityId = new EntityId('CAMPAIGN_ID',
> $campaignId);
>
> //Start Jobs
> $jobId =
> $this->createAndBeginBulkJob($scopingEntityId,
> $operations);
>
> // Monitor and retrieve results from job.
> $operationResultsByPart = $this-
>
> >retrieveResultsFromBulkJob($jobId);
>
> //Process results
> $results =
> $this->processResultsFromBulkJob($operationResultsByPart,
>
> 'AdGroupCriterion');
> }
> return $results;
> }
>
> function createAndBeginBulkJob($scopingEntityId, $operationsByPart)
> {
>
> $bulkMutateJobService = $this->adWordsUser-
>
> >GetBulkMutateJobService();
>
> // Initialize the bulk mutate job id.
> $jobId = NULL;
>
> // Note: A job may have no more than 100 Request Parts. Each
> Part
> may have
> // no more than 25 operation streams and no more than 10,000
> operations
> for ($partCounter = 0; $partCounter <
> count($operationsByPart);
> $partCounter++) {
> // Create operation stream.
> $opStream = new OperationStream();
> $opStream->scopingEntityId = $scopingEntityId;
> $opStream->operations =
> $operationsByPart[$partCounter];
>
> // Create bulk mutate request part.
> $part = new BulkMutateRequest();
> $part->partIndex = $partCounter;
> $part->operationStreams = array($opStream);
>
> // Create bulk mutate job.
> $job = new BulkMutateJob();
> $job->id = $jobId;
> $job->numRequestParts = sizeof($operationsByPart);
> $job->request = $part;
>
> // Create operation.
> $operation = new JobOperation();
> $operation->operand = $job;
>
> // If this is our first part, then the job must be
> added, not set.
> if ($partCounter == 0) {
> $operation->operator = 'ADD';
> } else {
> $operation->operator = 'SET';
> }
>
> try{
> // Add/set the job. The job will not start
> until all parts are
> added.
> $job =
> $bulkMutateJobService->mutate($operation);
> $jobId = $job->id;
> //echo "Job Id = $jobId\n";
> }catch (SoapFault $fault) {
> print_r($fault);
> //Extract error and place on fault stack
> $this->checkFault($fault);
> }
>
> // Store job id.
> $jobId = $job->id;
> }
>
> return $jobId;
> }
>
> function retrieveResultsFromBulkJob($jobId) {
>
> $bulkMutateJobService =
> $this->adWordsUser->GetBulkMutateJobService();
>
> $operationResultsByPart = array();
>
> // Create selector.
> $selector = new BulkMutateJobSelector();
> $selector->jobIds = array($jobId);
>
> // Loop while waiting for the job to complete.
> do {
> $jobs = $bulkMutateJobService->get($selector);
> $job = $jobs[0];
>
> print 'Bulk mutate job with id "' . $job->id . '" has
> status "' .
> $job->status . "\".\n";
>
> if (($job->status == 'PENDING') || ($job->status ==
> 'PROCESSING'))
> {
> sleep(10);
> }
> } while (($job->status == 'PENDING') || ($job->status ==
> 'PROCESSING'));
>
> if ($job->status == 'FAILED') {
> throw new ApiException(NULL, 'Job failed.');
> }
>
> for ($i = 0; $i < $job->numRequestParts; $i++) {
>
> // Set selector to retrieve results for part.
> $selector->resultPartIndex = $i;
> $jobsWithResult =
> $bulkMutateJobService->get($selector);
>
> // Get the operation results.
> $index = $jobWithResult->result->partIndex;
> $operationResultsByPart[$index] = $jobsWithResult[0];
> }
>
> return $operationResultsByPart;
> }
--
You received this message because you are subscribed to the Google Groups
"AdWords API Forum" group.
To post to this group, send email to adwords-...@googlegroups.com.
To unsubscribe from this group, send email to
adwords-api+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/adwords-api?hl=en.
