Scraping

Does anyone have a suggestion on ways to grab the actual rendered page programmatically? So, we have an internal page that loads text and assets via javascript and renders out into the DOM. I want to grab the rendered DOM, not the HTML source of the page.

1 Like

If you want the final page after Javascript have run, I reckon you can do so with Capybara and Poltergeist, however I have only done this for request specs.

1 Like

Watir will programmatically run your web browser and your browser will render everything. You can then access the DOM rendered in the browser in Ruby.

1 Like