NullPointerException on htmlunit.WebClient.getPage

[lrpowell]

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.html.HtmlPage.cleanUp(HtmlPage.java:360)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:592)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:456)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:541)
at com.ibm.as400.pt.viewer.script.RunPDIReport.exportDispatchServlet(RunPDIReport.java:477)
at com.ibm.as400.pt.viewer.script.RunPDIReport.doLoginAndExport(RunPDIReport.java:407)
at com.ibm.as400.pt.viewer.script.RunPDIReport.main(RunPDIReport.java:178)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:408)
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:377)
at com.gargoylesoftware.htmlunit.html.HtmlPage.cleanUp(HtmlPage.java:357)
... 7 more
Caused by: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.websocket.JettyWebSocketAdapter.closeClinet(JettyWebSocketAdapter.java:135)
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:405)
... 9 more

**The dispatch request is : **

WebRequest[<url="http://localhost:9080/Navigator/DispatcherServlet/perf/loadPerspectives", POST, EncodingType[name=application/x-www-form-urlencoded], [], {Cookie=JSESSIONID=0000zGsxZkz6VSlx5FVsoC1SgaW:8f560a86-b308-4406-8092-29d0c4eb66be, Accept=/, Accept-Encoding=gzip, deflate, Content-Type=application/json}, null>]

The code is:

	URL loadPerspUrl = new URL("http://"+getSystemName()+":9080/Navigator/DispatcherServlet/perf/loadPerspectives");
	WebRequest requestSettings = new WebRequest(loadPerspUrl, HttpMethod.POST);
	requestSettings.setAdditionalHeader("Content-Type", "application/json");
	requestSettings.setAdditionalHeader("Cookie", m_cookie_jsessionid);

	Page redirectPage = webClient.getPage(requestSettings);  // line 477 **exception occurs here **
	webClient.waitForBackgroundJavaScriptStartingBefore(10000);
		
	WebResponse response = redirectPage.getWebResponse();

[rbri]

can you please post the whole stack trace for this

[lrpowell]

From what I have found, I believe this is the full stack trace

• https://stackoverflow.com/questions/7597797/how-to-show-full-stack-trace-on-eclipse
• https://docs.oracle.com/javase/7/docs/api/java/lang/Throwable.html#printStackTrace%28%29
  If there is something else I need to do for more information, let me know.

[rbri]

You mean

.. 7 more
Caused by: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.websocket.JettyWebSocketAdapter.closeClinet(JettyWebSocketAdapter.java:135)
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:405)
... 9 more

this is the end of the trace?

[lrpowell]

ya, that is it. Maybe there is something else to get somehow? I see that the program does not complete, it stays active (red box still available to stop), but also no step or run available...

[rbri]

Ok, thanks, have to think about this....

[rbri]

Have added some synchronization - this might fix the problem.

[lrpowell]

Thanks! I unfortunately did not keep this code as it was for a test. I need to do that as a practice (keep copies of source as it stands for an open issue)

[rbri]

Thanks, no problem i take this as fixed and will see if this comes up again.

[grantwforsythe]

I started getting this exception after updating from version 2.34. Below is an example of the request:

final String UTF = "UTF-8";

try (WebClient webClient = new WebClient(BrowserVersion.CHROME)
{
	HtmlPage page = webClient.getPage("https://example.com");
	
	// Request parameters
	String username = "john.smith"
	String password = "password123"
	String token = page.getFirstByXPath("//*[@id='token']");
	String itemId = page.getFirstByXPath("//*[@id='itemId']");
	
	URL loadPerspUrl = new URL("https://example.com");
	WebRequest request = new WebRequest(url, HttpMethod.POST);
	
	Map<String, String> header = new HashMap<String, String>();
	// Setting all of the headers ...
	request.setAdditionalHeaders(header);

	String requestBody = String.format(
		"Username=%s&Password=%s&Token=%s&ItemId=%s",
		URL.encode(username, UTF),
		URL.encode(password, UTF), 
		URL.encode(token, UTF), 
		URL.encode(itemId, UTF)
        );

	// This request will return an xhr document
	webClient.getPage(requestSettings);  // **exception occurs here **
}

The exception is being thrown intermittently as I am signing in and out of multiple accounts. The first time making the request always fails.

[rbri]

@grantwforsythe do you have a page for me to test this?
Can you please post your whole stack trace.

[grantwforsythe]

Hi @rbri. Apologies for the late reply. The above code is from a work-related problem so I am not sure how much I can disclose but the stack trace was very similar to @lrpowell's.

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.html.HtmlPage.cleanUp(HtmlPage.java:360)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:592)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:456)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:368)
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:541)
at com.ibm.as400.pt.viewer.script.RunPDIReport.exportDispatchServlet(RunPDIReport.java:477)
at com.ibm.as400.pt.viewer.script.RunPDIReport.doLoginAndExport(RunPDIReport.java:407)
at com.ibm.as400.pt.viewer.script.RunPDIReport.main(RunPDIReport.java:178)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:408)
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:377)
at com.gargoylesoftware.htmlunit.html.HtmlPage.cleanUp(HtmlPage.java:357)
... 7 more
Caused by: java.lang.NullPointerException
at com.gargoylesoftware.htmlunit.websocket.JettyWebSocketAdapter.closeClinet(JettyWebSocketAdapter.java:135)
at com.gargoylesoftware.htmlunit.javascript.host.WebSocket.close(WebSocket.java:405)
... 9 more

After cleaning up the code base, I was able to resolve the issue by wrapping the POST request in a try-catch block. If any exceptions were thrown, it would try making the request again which solved the problem.

[rbri]

closeClinet - this typo indicates your stack trace is from a version before i made the fix for this.
Pleae make sure you have at least version 2.49.1 in use. And of course using the latest release 2.62.0 is always preferred.