Errors thrown when inner frame is not available

[jogger2510]

My site runs a DNS blocker filtering the regex "betsonsport.ru", returning NXDOMAIN for any query. When trying to load content from this domain into a frame, regular browsers stay silent and display white space or a warning sign instead.

HtmlUnit throws an error that cannot be suppressed with all the error silencers. In my view, no connection attempt should be made at all.

Error snippet:
Mai 16, 2025 12:59:32 PM org.htmlunit.html.BaseFrameElement loadInnerPageIfPossible
SCHWERWIEGEND: IOException when getting content for iframe: url=[https://www.betsonsport.ru/banners/lend4.php]
javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:383)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1326)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1203)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1146)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:447)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:206)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1507)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1422)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436)
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384)
at org.htmlunit.httpclient.HtmlUnitSSLConnectionSocketFactory.connectSocket(HtmlUnitSSLConnectionSocketFactory.java:176)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at org.htmlunit.HttpWebConnection.getResponse(HttpWebConnection.java:191)
at org.htmlunit.WebClient.getWebResponseOrUseCached(WebClient.java:1726)
at org.htmlunit.WebClient.loadWebResponseFromWebConnection(WebClient.java:1631)
at org.htmlunit.WebClient.loadWebResponse(WebClient.java:1556)
at org.htmlunit.WebClient.getPage(WebClient.java:485)
at org.htmlunit.WebClient.getPage(WebClient.java:402)
at org.htmlunit.html.BaseFrameElement.loadInnerPageIfPossible(BaseFrameElement.java:211)
at org.htmlunit.html.BaseFrameElement.loadInnerPage(BaseFrameElement.java:121)
at org.htmlunit.html.HtmlPage.loadFrames(HtmlPage.java:1898)
at org.htmlunit.html.HtmlPage.initialize(HtmlPage.java:267)
at org.htmlunit.WebClient.loadWebResponseInto(WebClient.java:701)
at org.htmlunit.WebClient.loadWebResponseInto(WebClient.java:575)
at org.htmlunit.WebClient.getPage(WebClient.java:493)
at org.htmlunit.WebClient.getPage(WebClient.java:402)
at org.htmlunit.WebClient.getPage(WebClient.java:538)
at org.htmlunit.WebClient.getPage(WebClient.java:520)

[rbri]

@jogger2510 as a workaround you can implement a content blocker for HtmlUnit - https://www.htmlunit.org/details.html#Content_blocking

Will have a look and (hopefully) fix your case

[rbri]

the related source code looks like this

try {
    webClient.getPage(enclosedWindow_, request);
}
catch (final IOException e) {
    if (LOG.isErrorEnabled()) {
        LOG.error("IOException when getting content for " + getTagName() + ": url=[" + url + "]", e);
    }
}

I think the code is fine (and you see the error in your log). The processing of the page does not stop, only the error is logged.
You can configure your logger to ignore these kind of messages (https://www.htmlunit.org/logging.html)

[rbri]

and btw. you might see someting similar in your browser console

[jogger2510]

Your advice is helpful, but I do not fully agree.
Safari logs: "Failed to load resource: Invalid URL" for a bunch a blocked host names including the one above, some blocked by the local machine, some blocked elsewhere in the network. Obviously the host name is checked before any connection attempt is made.
I think some code should be added, like
try { InetAddress.getByName(new URL(url).getHost()); } catch (UnknownHostException uhe) { log_error_and_return(); }

[jogger2510]

The frame handler override did not work for me. The following quick hack worked:

            webClient = new WebClient() {
                @Override
                public <P extends Page> P getPage(final URL url) throws IOException, FailingHttpStatusCodeException {
                    try {
                        InetAddress.getByName(url.getHost());
                    } catch (Exception e) { throw new IOException(); }
                    return super.getPage(url);
                }
            };

[rbri]

Obviously the host name is checked before any connection attempt is made.

Did not really get your point.

As far as i can see, HtmlUnit does NOT throw an error - the error is catched and logged, nothing more. Or do you really get that error?

What do you expect from trying to resolve the name first?

[jogger2510]

This is about where it occurs and how it is logged. Obviously this type of error occurs at an unexpected place in code.

• The error is NOT caught by the error silencers. No log entry with my quick hack.
• The error actually causes three stack traces printed per occurence (I only posted the first few lines)
• The error message from the stack trace "SSLHandshakeException: (certificate_unknown) PKIX path building failed" is totally misleading. UnknownHostException should be logged.
• Personally I would prefer a sanity check on the parameters before Apache blowing everything up in a SSL handshake

[rbri]

What do you mean by 'error silencers'?

[jogger2510]

Good point. The error before my fix occurs in Apache code and I have SEVERE enabled there. My fix keeps error handling within HtmlUnit.

[rbri]

I fear your InetAddress.getByName(url.getHost()); might have some impact on the performance.

[jogger2510]

All OSes have local DNS caches, so there will be no extra DNS traffic. OTOH, any page that has an element blocked will be faster, because that Apache SSL stuff does not come for free. My network has 165,000+ hosts blocked, the blocklist strikes about four times per web page. Nearly everybody will have some kind of blocking security software installed to protect against malware, tracking, advertising etc.

[rbri]

Sounds reasonable, but i will do the 4.12 release first and then try your suggestion.