On Thu, Dec 14, 2023 at 11:40 AM Anuj Bhargava <anujb...@gmail.com> wrote:
> solr.in.sh has the following entry -
> SOLR_JAVA_MEM="-Xms15g -Xmx15g"
> SOLR_OPTS="$SOLR_OPTS -Dsolr.disable.shardsWhitelist=true"
>
There could be a possible memory leak in your application. You need to
profile your application (possibly with a Memory-Debugger)
>
> In which file should I define SOLR_HEAP and what should be the ideal value.
> My server has 128GB Ram and 2TB SSD
>
> Regards,
>
> Anuj
>
> On Thu, 14 Dec 2023 at 10:56, Deepak Goel <deic...@gmail.com> wrote:
>
> > On Thu, 14 Dec 2023, 10:11 Anuj Bhargava, <anujb...@gmail.com> wrote:
> >
> > > There are some files in /var/solr/logs. For example -
> > > solr_oom_killer-8983-2023-12-14_05_30_42.log
> > >
> > > and this contains -
> > >
> > > Running OOM killer script for process 2861377 for Solr on port 8983
> > > Killed process 2861377
> > >
> >
> > Looks like you are running out of memory. Increase your heap size
> >
> >
> > > On Wed, 13 Dec 2023 at 17:36, Anuj Bhargava <anujb...@gmail.com>
> wrote:
> > >
> > > > Last Modified:about 7 hours ago
> > > > Num Docs:803749
> > > > Max Doc:803749
> > > > Heap Memory Usage:271668
> > > > Deleted Docs:0
> > > > Version:3789
> > > > Segment Count:25
> > > > Current:
> > > >
> > > > Instance
> > > > CWD:/opt/solr-8.11.2/server
> > > > Instance:/var/solr/data/gt128_dga
> > > > Data:/var/solr/data/gt128_dga/data
> > > > Index:/var/solr/data/gt128_dga/data/index
> > > > Impl:org.apache.solr.core.NRTCachingDirectoryFactory
> > > >
> > > > On Wed, 13 Dec 2023 at 16:31, Deepak Goel <deic...@gmail.com> wrote:
> > > >
> > > >> On Wed, Dec 13, 2023 at 1:51 PM Anuj Bhargava <anujb...@gmail.com>
> > > wrote:
> > > >>
> > > >> > 12/13/2023, 11:04:12 AM ERROR false x:gt128_dga UpdateLog
> > > >> > java.io.EOFException
> > > >> >
> > > >>
> > > >> You are trying to update a file which does not exist. We need more
> > > details
> > > >> about the file which you are trying to update.
> > > >>
> > > >>
> > > >> > 12/13/2023, 11:04:13 AM WARN false x:gt128_dga UpdateLog Log
> replay
> > > >> > finished. recoveryInfo=RecoveryInfo{adds=82717 deletes=0
> > > deleteByQuery=0
> > > >> > errors=0 positionOfStart=0}
> > > >> >
> > > >> > On Wed, 13 Dec 2023 at 13:38, Deepak Goel <deic...@gmail.com>
> > wrote:
> > > >> >
> > > >> > > On Wed, Dec 13, 2023 at 1:35 PM Anuj Bhargava <
> anujb...@gmail.com
> > >
> > > >> > wrote:
> > > >> > >
> > > >> > > > Noticed that whenever large amount of data is being indexed it
> > > >> stops.
> > > >> > > >
> > > >> > > >
> > > >> > > What is the error it spits out?
> > > >> > >
> > > >> > >
> > > >> > > > Configuration is -
> > > >> > > >
> > > >> > > > <dataSource type="JdbcDataSource"
> > > >> > > > driver="com.mysql.jdbc.Driver"
> > > >> > > > batchSize="-1"
> > > >> > > > autoReconnect="true"
> > > >> > > > useSSL="false"
> > > >> > > > socketTimeout="0"
> > > >> > > > connectTimeout="0"
> > > >> > > > encoding="UTF-8"
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> url="jdbc:mysql://xxx.xxx.xx.xx:3306/globe-data?zeroDateTimeBehavior=convertToNull"
> > > >> > > > user="globe-user"
> > > >> > > > password="globe-password"/>
> > > >> > > > <document>
> > > >> > > >
> > > >> > > > *My jetty.xml file is* -
> > > >> > > >
> > > >> > > > <New id="httpConfig"
> > > >> > class="org.eclipse.jetty.server.HttpConfiguration">
> > > >> > > > <Set name="secureScheme">https</Set>
> > > >> > > > <Set name="securePort"><Property
> > name="solr.jetty.secure.port"
> > > >> > > > default="8443" /></Set>
> > > >> > > > <Set name="outputBufferSize"><Property
> > > >> > > > name="solr.jetty.output.buffer.size" default="32768" /></Set>
> > > >> > > > <Set name="outputAggregationSize"><Property
> > > >> > > > name="solr.jetty.output.aggregation.size" default="32768" /><$
> > > >> > > > <Set name="requestHeaderSize"><Property
> > > >> > > > name="solr.jetty.request.header.size" default="65536" /></Set>
> > > >> > > > <Set name="responseHeaderSize"><Property
> > > >> > > > name="solr.jetty.response.header.size" default="32768"
> /></Set>
> > > >> > > > <Set name="sendServerVersion"><Property
> > > >> > > > name="solr.jetty.send.server.version" default="false" /></Set>
> > > >> > > > <Set name="sendDateHeader"><Property
> > > >> > > name="solr.jetty.send.date.header"
> > > >> > > > default="false" /></Set>
> > > >> > > > <Set name="headerCacheSize"><Property
> > > >> > > > name="solr.jetty.header.cache.size" default="1024" /></Set>
> > > >> > > > <Set name="delayDispatchUntilContent"><Property
> > > >> > > > name="solr.jetty.delayDispatchUntilContent" default="false"
> > > /></Set>
> > > >> > > > <!-- Uncomment to enable handling of X-Forwarded- style
> > > headers
> > > >> > > >
> > > >> > > > On Wed, 13 Dec 2023 at 12:08, Deepak Goel <deic...@gmail.com>
> > > >> wrote:
> > > >> > > >
> > > >> > > > > On Wed, Dec 13, 2023 at 11:03 AM Anuj Bhargava <
> > > >> anujb...@gmail.com>
> > > >> > > > wrote:
> > > >> > > > >
> > > >> > > > > > The solr stops too often and I have to restart it. Is
> there
> > a
> > > >> > script
> > > >> > > > > which
> > > >> > > > > > checks if solr is running and if not it starts
> > automatically.
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > > You need to find why Solr is stopping too often. Restarting
> it
> > > is
> > > >> > not a
> > > >> > > > > graceful option.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > >
> > > >> > > > > > Regards,
> > > >> > > > > >
> > > >> > > > > > Anuj
> > > >> > > > > >
> > > >> > > > > > On Tue, 12 Dec 2023 at 23:29, Shawn Heisey
> > > >> > > <apa...@elyograg.org.invalid
> > > >> > > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > On 12/11/23 21:25, Anuj Bhargava wrote:
> > > >> > > > > > > > http://xxx.xxx.xx.xxx:8983/solr/#/
> > > >> > > > > > > >
> > > >> > > > > > > > I often get the following error -
> > > >> > > > > > > >
> > > >> > > > > > > > This site can’t be reached xxx.xxx.xx.xxx refused to
> > > >> connect.
> > > >> > > > > > > > Try:
> > > >> > > > > > > >
> > > >> > > > > > > > Checking the connection
> > > >> > > > > > > > Checking the proxy and the firewall
> > > >> > > > > > > > ERR_CONNECTION_REFUSED
> > > >> > > > > > > >
> > > >> > > > > > > > 8983 port is open
> > > >> > > > > > >
> > > >> > > > > > > There's not a lot of info here. Starting in version
> > 9.0.0,
> > > >> Solr
> > > >> > > only
> > > >> > > > > > > listens on localhost by default. To listen on all
> > > interfaces,
> > > >> > set
> > > >> > > > the
> > > >> > > > > > > SOLR_JETTY_HOST environment variable to 0.0.0.0 in the
> > > include
> > > >> > > > script.
> > > >> > > > > > >
> > > >> > > > > > > Thanks,
> > > >> > > > > > > Shawn
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > Deepak
> > > >> > > > > "The greatness of a nation can be judged by the way its
> > animals
> > > >> are
> > > >> > > > treated
> > > >> > > > > - Mahatma Gandhi"
> > > >> > > > >
> > > >> > > > > +91 73500 12833
> > > >> > > > > deic...@gmail.com
> > > >> > > > >
> > > >> > > > > LinkedIn: www.linkedin.com/in/deicool
> > > >> > > > > Upwork:
> > https://www.upwork.com/freelancers/~0150257dd6dfd15b4b
> > > >> > > > >
> > > >> > > > > "Plant a Tree, Go Green"
> > > >> > > > >
> > > >> > > > > Make In India : http://www.makeinindia.com/home
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>
