How to avoid intermittent errors on capybara and selenium
Running selenium specs sometimes it might happen that the test suite is green in our computer, but in the integration server (I've also experienced this issue on Travis) it randomly fails with Capybara ElementNotFound errors or similar, for example:
Failure/Error: text = page.find("##{id} input").value
Capybara::ElementNotFound:
Unable to find css "#best_in_place_user_1210_address input"
It has been a hard issue to track down, because the real error is a timeout error. It happens that, when executing javascript tests, capybara waits some time before throwing an error (like element not found). When the connection between selenium and firefox is too slow, maybe due to the high system load or anything else, the firefox instance may not respond in time and capybara fails the expectation.
Incrementing the default timeout may increase the success rate (setting Capybara.default_wait_time to some big number) but it does not solve the issue in any case. I've been googling for this issue and I haven't found any solution, so I come up with a simple one: Retry the test if it fails due to timeout or element not found errors. Add this snippet to some support file:
def retry_on_timeout(n = 3, &block)
block.call
rescue Capybara::TimeoutError, Capybara::ElementNotFound => e
if n > 0
puts "Catched error: #{e.message}. #{n-1} more attempts."
retry_on_timeout(n - 1, &block)
else
raise
end
end
Then in your failing spec wrap the code in a retry_on_timeout call:
retry_on_timeout do
# your test
end
By default it will try the test for 3 times, you can override it by calling it like so: retry_on_timeout(5)