I work at CircleCI, so I can take a stab at some of these questions.
Selenium tests are a pain, and we come across a lot of customers who have unstable selenium tests. We're building a lot in the next few months to help with it.
Firstly, one feature that we've just put into beta is to show you failing tests. If you're using RSpec or Cucumber or another framework that can generate junit.xml files, we should be able to show you which tests failed, which is shown on the build page, in chat integrations, etc. (Also available via API if you want to consume it via hubot or at the command line or similar).
We're planning to use this to track flaky tests. Someone on our team has the objective to build it this year, so if things go as we optimistically guess that they might, we might see it this year. Otherwise, it should be easy to use the API to build that yourselves.
Thanks! I'll have to check that out. I really love the CircleCI service - incredible time saver in getting continuous integration setup and so many features out of the box. Would definitely recommend it to anybody looking for a hosted CI solution (good price, too!).
Selenium tests are a pain, and we come across a lot of customers who have unstable selenium tests. We're building a lot in the next few months to help with it.
Firstly, one feature that we've just put into beta is to show you failing tests. If you're using RSpec or Cucumber or another framework that can generate junit.xml files, we should be able to show you which tests failed, which is shown on the build page, in chat integrations, etc. (Also available via API if you want to consume it via hubot or at the command line or similar).
We're planning to use this to track flaky tests. Someone on our team has the objective to build it this year, so if things go as we optimistically guess that they might, we might see it this year. Otherwise, it should be easy to use the API to build that yourselves.
It seems like that might help you - let me know!