Is Selenium testing good or bad?
I've just read this post of Jay Fields about Selenium. I have some different points about Selenium testing and its use, I'm owning a post about Selenium so the time came.
First of all, I'm not a experienced Selenium tester, I've heard about it and waited until have some time to put the tests on a legacy application that I support. Let's put on context, Selenium for me was hard to learn. Jay says that is a easy to learn but wasn't for me. Couple of reasons:
The Selenium IDE is easy but limited, so I went for the big fish SeleniumRC.
My legacy application is very bad implemented. About all pages has invalid HTML syntax and JavaScript errors, and the application has a lot of pages and interactions.
As Jay points out, Selenium has a lot of languages you can write tests and we can get a little confused about what language to choose to write them. This can be a problem but I don't think so, if you think a little why you should choose a language in favor of another. You have just to make the decision of what language suites you better, can be the language you are most familiar or, as I decided, the language of the implementation of your application. I've chosen the second one because my team writes in Java, my app was written in Java and I think is a good thing to keep your tests with your application in the repository. Java has other benefits, typing, with Perl I had to check all return types and look at the API all the time. But with some practice you get how Selenium works and you get to write tests more rapidly.
I really don't know about other experiences with Selenium as Jay mention, if developers had to throw their tests away after certain size, this still is not a problem for me, maybe in the future Selenium bites me, still...
Jay argues that Selenium tests are slow and browsers are buggy, it is absolute true. A simple suite will run in 10 minutes or so, and that's slow. But I don't think this is a Selenium problem, in fact Selenium is fast where it can, surely more fast than written down scripts for users to test the application. Browsers. Oh! Browsers... we developers still fell a major pain when someone shouts "It doesn't work for me!" The browser war made a lot of collateral damage, but still it's not Selenium fault. In fact Selenium came to help us with these damages. Being capable of programatically test my application on a Firefox and IE is a bless.
When a test fail little information is provided, as Jay puts, and it's true. I agree that if more information was provided will be a big gain, but this is not, as far as I know, Selenium's fault. We have to remember that we are testing the final product, the one which is stripped of "business logic" (i know that the behavior is business logic but I'm talking about of the business logic we hid in our classes), in this case we're experiencing the user environment and it's a harsh one. I develop for the web for about nine years and some problems in the user environment are really bizarre.
In my opinion Selenium is a excellent initiative and has much value, of course that is not the only testing you should do. When using Selenium you are able to catch some very complex interactions that a simple unit testing will not, by the fact that "in pieces" the application behaves right but when the user go from one state to another and back again some missing parameter, a misplaced element on a page and etc, can bite you.
For who wrote such tests using LWP library in Perl and looking for elements using DOM parsers I think Selenium is great. Can be better? Of course! I had a difficult time in find complete examples and I think that the docs still need improvement but I've found my way through these and the faulty, sloppy made application can be tested and refactored without noticing a break after weeks of development, and that's good for me. I can work more confident on this application then before, this is a good sign.