[ https://issues.apache.org/jira/browse/SOLR-15417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17350029#comment-17350029 ]
Takashi Sasaki commented on SOLR-15417: --------------------------------------- I started the server in SolrCloud and ran the code, waited about 10 minutes but couldn't reproduce it, so I reloaded the collection and it did! Code: {code:java} import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Optional; import org.apache.solr.client.solrj.SolrClient; import org.apache.solr.client.solrj.impl.CloudSolrClient; import org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient; import org.apache.solr.client.solrj.impl.HttpSolrClient; import org.apache.solr.client.solrj.request.UpdateRequest; import org.apache.solr.client.solrj.response.UpdateResponse; import org.apache.solr.common.SolrInputDocument; import static java.lang.System.*; public class Reproduce { public static void main(String[] args) throws Exception { // SolrClient solrClient = new HttpSolrClient.Builder().withBaseSolrUrl("http://localhost:8983/solr/techproducts").build(); // SolrClient solrClient = new ConcurrentUpdateSolrClient.Builder("http://localhost:8983/solr/techproducts").withThreadCount(4).withQueueSize(500).build(); CloudSolrClient solrClient = new CloudSolrClient.Builder(List.of("http://localhost:8983/solr")).build(); solrClient.setDefaultCollection("techproducts"); List<String> idList = List.of("TWINX2048-3200PRO", "VS1GB400C3", "VDBDB1A16", "MA147LL/A", "F8V7067-APL-KIT"); List<SolrInputDocument> batch = new ArrayList<>(); for(int idx = 1; idx <= idList.size(); idx++) { SolrInputDocument doc = new SolrInputDocument(); if (idx == 3) { doc.addField("id", idList.get(idx - 1) + "_invalid"); } else { doc.addField("id", idList.get(idx - 1)); } doc.addField("hasUserAssertions", new HashMap<String, Object>() {{ put("set", true); }}); // this makes sure update only succeeds when record with specified id exists doc.addField("_version_", 1); out.println("Added solr doc for record: " + doc.get("id")); batch.add(doc); } UpdateRequest updateRequest = new UpdateRequest(); updateRequest.setAction(UpdateRequest.ACTION.COMMIT, false, false); updateRequest.setParam("failOnVersionConflicts", "false"); updateRequest.add(batch); // List<SolrInputDocument> batch updateRequest.lastDocInBatch(); try { UpdateResponse process = updateRequest.process(solrClient); out.println("xhk205 process = " + process.toString()); } catch (Exception e) { out.println("Failed to update solr doc, error message: " + e.getMessage()); } } } {code} Output: {code:java} Added solr doc for record: id=TWINX2048-3200PRO Added solr doc for record: id=VS1GB400C3 Added solr doc for record: id=VDBDB1A16_invalid Added solr doc for record: id=MA147LL/A Added solr doc for record: id=F8V7067-APL-KIT Failed to update solr doc, error message: Error from server at http://192.168.0.7:8983/solr/techproducts_shard1_replica_n1: Document not found for update. id=VDBDB1A16_invalid {code} Query: [http://localhost:8983/solr/techproducts/select?fq=hasUserAssertions:true&q=*:*] {code:java} {responseHeader: {zkConnected: true,status: 0,QTime: 2,params: {q: "*:*",fq: "hasUserAssertions:true"}},response: {numFound: 2,start: 0,docs: [{id: "TWINX2048-3200PRO",name: "CORSAIR XMS 2GB (2 x 1GB) 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) Dual Channel Kit System Memory - Retail",manu: "Corsair Microsystems Inc.",manu_id_s: "corsair",cat: ["electronics","memory"],features: ["CAS latency 2, 2-3-3-6 timing, 2.75v, unbuffered, heat-spreader"],price: 185,price_c: "185.0,USD",popularity: 5,inStock: true,store: "37.7752,-122.4232",manufacturedate_dt: "2006-02-13T15:26:37Z",payloads: "electronics|6.0 memory|3.0",hasUserAssertions: true,_version_: 1700551118287798300,price_c____l_ns: 18500},{id: "VS1GB400C3",name: "CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM Unbuffered DDR 400 (PC 3200) System Memory - Retail",manu: "Corsair Microsystems Inc.",manu_id_s: "corsair",cat: ["electronics","memory"],price: 74.99,price_c: "74.99,USD",popularity: 7,inStock: true,store: "37.7752,-100.0232",manufacturedate_dt: "2006-02-13T15:26:37Z",payloads: "electronics|4.0 memory|2.0",hasUserAssertions: true,_version_: 1700551118294089700,price_c____l_ns: 7499}]}} {code} As you said, the process is interrupted when the exception is thrown, and only the first two documents are updated. From reading the reference document, it does indeed seem to be a bug. I'm not sure if it's an implicit constraint or not, so I'll look into the details of how the option was created. > exception in updateRequest caused all subsequent update fail > ------------------------------------------------------------ > > Key: SOLR-15417 > URL: https://issues.apache.org/jira/browse/SOLR-15417 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: UpdateRequestProcessors > Affects Versions: 8.5.1 > Reporter: xuanyu huang > Priority: Minor > > Hi there, > I'm using solrj 8.8.2 for a 8.5.1 solr server. I have a list of records and > in a for loop I construct an updateRequest to update each record. > Code looks like this > {code:java} > for (Map<String, Object> map : maps) { > if (map.containsKey("record_uuid")) { > UpdateRequest updateRequest = new UpdateRequest(); > updateRequest.setAction( UpdateRequest.ACTION.COMMIT, false, false); > SolrInputDocument doc = new SolrInputDocument(); > if (idx == 3) { > doc.addField("id", map.get("record_uuid") + "_invalid"); > } else { > doc.addField("id", map.get("record_uuid")); > } > idx++; > doc.addField("hasUserAssertions", new HashMap<String, Object>() {{ > put("set", true); }}); > // this makes sure update only succeeds when record with specified id > exists > doc.addField("_version_", 1); > logger.debug("Added solr doc for record: " + doc.get("id")); > updateRequest.add(doc); > try { > updateRequest.setParam("failOnVersionConflicts", "false"); > UpdateResponse process = updateRequest.process(solrClient); > System.out.println("xhk205 process = " + process.toString()); > } catch (Exception e) { > logger.error("Failed to update solr doc, error message: " + > e.getMessage(), e); > } > }{code} > There are 5 requests in total and I intentionally set the id in 3rd request > to be an invalid id so that updateRequet for 3rd record should fail. (This is > to mimic the situation where the record to be updated no longer exists in > solr, so I only want those updates with a valid id to succeed, those updates > with an invalid id should fail/rejected instead of creating a new reocrd in > solr, so I used __version__=1). > > Also I used the syntax to do partial update. > The variable doc looks like this > {code:java} > { > "id":"2d4b625d-8809-461f-b19b-d0c963e038ed", > "hasUserAssertions":{"set":true} > } > {code} > > {color:#de350b}Since each update is put into its own request, I suppose only > the 3rd request will fail because there's no record with that id and I've set > __version__{color} {color:#de350b}to 1. But the reality is, only the first 2 > records were updated and other 3 not.{color} > {color:#de350b}When I queried in solr admin console after the update, with > [http://localhost:8983/solr/biocache/select?fq=hasUserAssertions:true&q=*:*] > there were only 2 records returned instead of 4.{color} > > Below is the log of IntelliJ IDEA: > > {code:java} > - Added solr doc for record: id=429cfa88-2e18-46b0-ab9f-f4efd9e36c3c > xhk205 process = {NOTE=the request is processed in a background stream} > - Added solr doc for record: id=5a80561b-a68d-46a3-a59b-03d267f35d0e > xhk205 process = {NOTE=the request is processed in a background stream} > - Added solr doc for record: id=ff2dcbee-9c05-491f-91a8-9f1fec348546_invalid > xhk205 process = {NOTE=the request is processed in a background stream} > - Added solr doc for record: id=baf7af1f-1525-403a-95bf-e28e432f1b12 > xhk205 process = {NOTE=the request is processed in a background stream} > - Added solr doc for record: id=4ea76605-c262-409b-845e-213f11ea4e34 > xhk205 process = {NOTE=the request is processed in a background stream}{code} > {code:java} > 2021-05-19 14:12:16,827 ERROR: [ConcurrentUpdateSolrClient] - error > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error > from server at http://localhost:8983/solr/biocache: Conflict request: > http://localhost:8983/solr/biocache/update?commit=true&softCommit=false&waitSearcher=false&failOnVersionConflicts=false&wt=javabin&version=2 > Remote error message: Document not found for update. > id=ff2dcbee-9c05-491f-91a8-9f1fec348546_invalid at > org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:394) > at > org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:191) > at > org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0{code} > > > {color:#de350b}The 3rd update obviously caused an exception. But why 4th and > 5th updates didn't succeed? Is it possible that this exception caused solr > client or server in some non-useable state so all subsequent updates > failed?{color} > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org