Scroll event to get pagination progress

[kronos72it]

I successfull done login with HtmlUnit on SITE, run a form search and got a page with results. Now in real world, to get the whole page result I need to scroll the page more times . But I am using HtmlUnit ,then I have done :

List

res=page.getByXPath(...) // got 13 items
ScriptResult res=page.executeJavaScript("window.scrollBy (0,2000);");
res=page.getByXPath(...) // would get more items than 13 items

But the 2nd XPath query return same result items as the first . Instead ,on browser, the scroll work well and return the second train of result items 8MORE THAN 13). Then , does HtmlUnit is not designed for this ? a bug ? or there is another HtmlUnit trick?

I tried also

ScriptResult res =page.getBody().fireEvent("scroll");

..But even this work .

Finally , I thought , if HtmlUnit can't manage Scroll Event
maybe is possible to fire the JS code fired by Scroll Event , in this way I can bypass the scroll event and obtaining his result .

But , does HtmlUnit can make an Ajax request ? How ?

thanks

[rbri]

Ok, and welcome :-)

Lets start simple:

• Do you use the latest version of HtmlUnit?
• Do you use HtmlUnit directly or the HtmlUnit Driver?
• Is the site you like to interact with public - can i access the site?

Regarding

ScriptResult res =page.getBody().fireEvent("scroll");

• you are sure the event listener that updates the page are assigned to the body?
• As you talk about Ajax. Have you added some kind of wait code after fireing the event? Usually all the updates are async so adding some wait might help already.

[kronos72it]

• ver 2.45 , there is now the 2.46 I will try it ...

• I don't know HtmlUnit Driver , I have to study it

• digiland.libero.it it's public ,yes

• you question is matter, it is the window object registering the listenere window.addEventListener(...)
  but there isn't a way to do webClinet.getCurrentWindow().fireEvent("scroll");
  I also tried page.executeJavaScript("window.scrollBy (0,2000);");

• after the call ,
  ScriptResult res=page.executeJavaScript("window.scrollBy (0,2000);");
  to tring invoke JS, I put

            `webClient.waitForBackgroundJavaScript(10000);`
  

[rbri]

Have done a first update - window.scroll()/scrollTo()/scrollBy() now triggers a scroll event

[rbri]

made some changed to the code

ScriptResult res=page.executeJavaScript("window.scrollBy (0,2000);");
webClient.waitForBackgroundJavaScript(10_000);

should trigger now at least one scroll event. Please use the latest SNAPSHOT build to test this.

[rbri]

The fix for this is part of the 2.47.11 release. Will close this. Feel free to open a new one if you face more problem.

Thanks for the report.