Hi, I'm trying to use selenium (solr 6.0.0 & nutch 1.11 & firefox 47.0) to parse javascript pagesI'm using this configuration for nutch-site:<property> <name>plugin.includes</name> <value>protocol-(httpclient|interactiveselenium|selenium)|urlfilter-(automaton|regex)|parse-(metatags|ext|html|js|swf|tika|zip)|index-(metadata|basic|anchor|geoip|dummy|links|more|replace|static)|scoring-opic|indexer-solr|urlnormalizer-(pass|regex|basic|ajax)|creativecommons|feed|headings|language-identifier|lib-nekohtml|lib-xml|microformats-reltag|mimetype-filter|nutch-extensionpoints|lib-selenium|subcollection|tld|parserfilter-naivebayes</value> <description>...</description> </property> and this configuration for parse-plugins.xml <parse-plugins>
<!-- by default if the mimeType is set to *, or if it can't be determined, use parse-tika --> <mimeType name="*"> <plugin id="parse-metatags"/> <plugin id="protocol-interactiveselenium"/> <plugin id="protocol-selenium"/> <plugin id="lib-selenium"/> <plugin id="nutch-extensionpoints"/> <plugin id="parse-js"/> <plugin id="parse-tika" /> <plugin id="feed"/> <plugin id="parse-html"/> <plugin id="parse-js"/> <plugin id="parse-html" /> </mimeType> <mimeType name="application/rss+xml"> <plugin id="parse-tika" /> <plugin id="feed" /> </mimeType> <mimeType name="application/x-bzip2"> <!-- try and parse it with the zip parser --> <plugin id="parse-zip" /> </mimeType> <mimeType name="application/x-gzip"> <!-- try and parse it with the zip parser --> <plugin id="parse-zip" /> </mimeType> <mimeType name="application/x-javascript"> <plugin id="parse-js" /> <plugin id="protocol-interactiveselenium"/> <plugin id="protocol-selenium"/> <plugin id="lib-selenium"/> <plugin id="nutch-extensionpoints"/> <plugin id="parse-metatags"/> <!--<plugin id="parse-ext"/>--> <plugin id="parse-tika" /> </mimeType> <mimeType name="application/x-shockwave-flash"> <plugin id="parse-swf" /> </mimeType> <mimeType name="application/zip"> <plugin id="parse-zip" /> </mimeType> <!--<mimeType name="text/html"> <plugin id="parse-html" /> </mimeType>--> <mimeType name="text/html"> <plugin id="parse-metatags"/> <plugin id="protocol-interactiveselenium"/> <plugin id="protocol-selenium"/> <plugin id="lib-selenium"/> <plugin id="nutch-extensionpoints"/> <!--<plugin id="parse-ext"/>--> <!--<plugin id="parse-js"/>--> <plugin id="parse-html" /> <plugin id="parse-tika" /> </mimeType> <mimeType name="application/xhtml+xml"> <plugin id="parse-metatags"/> <plugin id="protocol-interactiveselenium"/> <plugin id="protocol-selenium"/> <plugin id="lib-selenium"/> <plugin id="nutch-extensionpoints"/> <plugin id="parse-tika" /> <plugin id="feed" /> <plugin id="parse-html" /> </mimeType> <mimeType name="text/xml"> <plugin id="parse-metatags"/> <plugin id="protocol-interactiveselenium"/> <plugin id="protocol-selenium"/> <plugin id="lib-selenium"/> <plugin id="parse-tika" /> <plugin id="feed" /> </mimeType> The firefox window popup with a message about private browsing on it. However, I get the error below and the job crushes into flames: 17 18:44:13,029 INFO api.HttpRobotRulesParser - Couldn't get robots.txt for http://findjobs.mashable.com/: java.lang.RuntimeException: org.openqa.selenium.WebDriverException: Unable to bind to locking port 7054 within 45000 ms Build info: version: '2.48.2', revision: '41bccdd10cf2c0560f637404c2d96164b67d9d67', time: '2015-10-09 13:08:06' System info: host: 'solr', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-39-generic', java.version: '1.8.0_91' Driver info: driver.version: FirefoxDriver 2016-06-17 18:44:13,129 ERROR selenium.Http - Failed to get protocol output java.lang.RuntimeException: org.openqa.selenium.WebDriverException: Failed to connect to binary FirefoxBinary(/usr/bin/firefox) on port 7055; process output follows: ения Firefox для Ubuntu","creator":"Canonical Ltd.","homepageURL":null},{"locales":["sl"],"name":"Ubuntu Modifications","description":"Ubuntu razširitve za Firefox.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["sv-SE"],"name":"Ubuntu Modifications","description":"Ubuntu-paket för Firefox.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["uk"],"name":"Ubuntu Modifications","description":"Убунтівські доповнення до Firefox.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["zh-CN"],"name":"Ubuntu Modifications","description":"Ubuntu 火狐扩展包.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["zh-TW"],"name":"Ubuntu Modifications","description":"Ubuntu Firefox 擴充包。","creator":"Canonical Ltd.","homepageURL":null}],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"9.0","maxVersion":"37.0a1"}],"targetPlatforms":[],"multiprocessCompatible":false,"signedState":2,"seen":true} 1466178208570 DeferredSave.extensions.json DEBUG Save changes 1466178208570 addons.xpi DEBUG Updating database with changes to installed add-ons 1466178208570 addons.xpi-utils DEBUG Updating add-on states 1466178208571 addons.xpi-utils DEBUG Writing add-ons list 1466178208575 addons.xpi DEBUG Registering manifest for /usr/lib/firefox/browser/features/[email protected] 1466178208576 addons.xpi DEBUG Calling bootstrap method startup on [email protected] version 1.0.2 1466178208578 addons.xpi DEBUG Registering manifest for /usr/lib/firefox/browser/features/[email protected] 1466178208578 addons.xpi DEBUG Calling bootstrap method startup on [email protected] version 1.0 1466178208578 addons.xpi DEBUG Registering manifest for /usr/lib/firefox/browser/features/[email protected] 1466178208579 addons.xpi DEBUG Calling bootstrap method startup on [email protected] version 1.3.2 1466178208610 addons.manager DEBUG Registering shutdown blocker for XPIProvider 1466178208610 addons.manager DEBUG Provider finished startup: XPIProvider 1466178208610 addons.manager DEBUG Starting provider: LightweightThemeManager 1466178208611 addons.manager DEBUG Registering shutdown blocker for LightweightThemeManager 1466178208612 addons.manager DEBUG Provider finished startup: LightweightThemeManager 1466178208613 addons.manager DEBUG Starting provider: GMPProvider 1466178208621 addons.manager DEBUG Registering shutdown blocker for GMPProvider 1466178208622 addons.manager DEBUG Provider finished startup: GMPProvider 1466178208622 addons.manager DEBUG Starting provider: PluginProvider 1466178208622 addons.manager DEBUG Registering shutdown blocker for PluginProvider 1466178208622 addons.manager DEBUG Provider finished startup: PluginProvider 1466178208623 addons.manager DEBUG Completed startup sequence 1466178209011 addons.manager DEBUG Starting provider: <unnamed-provider> 1466178209011 addons.manager DEBUG Registering shutdown blocker for <unnamed-provider> 1466178209012 addons.manager DEBUG Provider finished startup: <unnamed-provider> 1466178209202 DeferredSave.extensions.json DEBUG Write succeeded 1466178209202 addons.xpi-utils DEBUG XPI Database saved, setting schema version preference to 17 1466178209202 DeferredSave.extensions.json DEBUG Starting timer 1466178209229 DeferredSave.extensions.json DEBUG Starting write 1466178209237 addons.repository DEBUG No addons.json found. 1466178209238 DeferredSave.addons.json DEBUG Save changes 1466178209242 DeferredSave.addons.json DEBUG Starting timer 1466178209309 addons.manager DEBUG Starting provider: PreviousExperimentProvider 1466178209310 addons.manager DEBUG Registering shutdown blocker for PreviousExperimentProvider 1466178209310 addons.manager DEBUG Provider finished startup: PreviousExperimentProvider 1466178209317 DeferredSave.addons.json DEBUG Starting write 1466178209329 DeferredSave.extensions.json DEBUG Write succeeded 1466178209357 DeferredSave.addons.json DEBUG Write succeeded (firefox:3352): Gtk-CRITICAL **: gtk_clipboard_set_with_data: assertion 'targets != NULL' failed Build info: version: '2.48.2', revision: '41bccdd10cf2c0560f637404c2d96164b67d9d67', time: '2015-10-09 13:08:06' System info: host: 'solr', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '3.19.0-39-generic', java.version: '1.8.0_91' Driver info: driver.version: FirefoxDriver at org.apache.nutch.protocol.selenium.HttpWebClient.getDriverForPage(HttpWebClient.java:118) at org.apache.nutch.protocol.selenium.HttpWebClient.getHtmlPage(HttpWebClient.java:155) at org.apache.nutch.protocol.selenium.HttpResponse.readPlainContent(HttpResponse.java:244) at org.apache.nutch.protocol.selenium.HttpResponse.<init>(HttpResponse.java:168) at org.apache.nutch.protocol.selenium.Http.getResponse(Http.java:56) at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:261) at org.apache.nutch.fetcher.FetcherThread.run(FetcherThread.java:290) Caused by: org.openqa.selenium.WebDriverException: Failed to connect to binary FirefoxBinary(/usr/bin/firefox) on port 7055; process output follows: ения Firefox для Ubuntu","creator":"Canonical Ltd.","homepageURL":null},{"locales":["sl"],"name":"Ubuntu Modifications","description":"Ubuntu razširitve za Firefox.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["sv-SE"],"name":"Ubuntu Modifications","description":"Ubuntu-paket för Firefox.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["uk"],"name":"Ubuntu Modifications","description":"Убунтівські доповнення до Firefox.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["zh-CN"],"name":"Ubuntu Modifications","description":"Ubuntu 火狐扩展包.","creator":"Canonical Ltd.","homepageURL":null},{"locales":["zh-TW"],"name":"Ubuntu Modifications","description":"Ubuntu Firefox 擴充包。","creator":"Canonical Ltd.","homepageURL":null}],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"9.0","maxVersion":"37.0a1"}],"targetPlatforms":[],"multiprocessCompatible":false,"signedState":2,"seen":true} 1466178208570 DeferredSave.extensions.json DEBUG Save changes 1466178208570 addons.xpi DEBUG Updating database with changes to installed add-ons 1466178208570 addons.xpi-utils DEBUG Updating add-on states 1466178208571 addons.xpi-utils DEBUG Writing add-ons list I have found some comments on this issue but nothing helpful: Remote driver & Firefox: Unable to bind to locking port 7054 within 45000 ms · Issue #7272 · SeleniumHQ/selenium-google-code-issue-archive | | | | | | | | | | | Remote driver & Firefox: Unable to bind to locking port 7054 within 45... Originally reported on Google Code with ID 7272 Hi All, I'm experiencing some sporadic issues with Remote ... | | | | In Firefox Browser:Unable to bind to locking port 7054 within 45000ms · Issue #6760 · SeleniumHQ/selenium-google-code-issue-archive | | | | | | | | | | | In Firefox Browser:Unable to bind to locking port 7054 within 45000ms · Iss... Originally reported on Google Code with ID 6760 selenium: 2.32.0, OS:Windows XP firefox version: 26.0. steps:... | | | | Unable to bind to locking port 7054 within 45000 ms : webdriver firefox | | | | | | | | | | | Unable to bind to locking port 7054 within 45000 ms : webdriver firefox i'm new to selenium webdriver i'm trying to run a simple test : i'm using firefox 17.0.1 and seleni... | | | | Please advice, Much obliged, Christian Fotache Tel: 0728.297.207
