Skip to content
December 19, 2013 / selenium34

Stand Alone Geb Utility

We needed a way for resources to get into automated testing quicker, and the biggest issue we have found with the majority of our resources is fear of the selector… What we wanted to give them was a quick and easy way to test out selectors and get a bit more immediate feedback. To do so we created the following:

Geb Command Practice Utility

This allows new resources to get in and immediately play with geb. You can basically do anything on the browser or a currently selected navigator. There are a couple of nuissances though:

  1. You must input a new url as go ‘http://www.fullsite.com’ (The initial command expects you to input a full url)
  2. Working off of the current navigator doesn’t always do what you’d expect. I found issues when I attempted to filter off of a Navigator which had previously been located

Beyond that you can do anything you would expect off of the browser / navigator objects.

August 26, 2013 / selenium34

Why are we using BrowserMob Proxy?

After reading the last post, you may be thinking, why are they running a proxy with PhantomJS. There are many reasons for us to run our tests through a proxy server. BrowserMob Proxy gives us many options on customizing our test runs, some of the features worth looking into:

  • We can artificially slow down our runs to mimic dial up connection users or those who might be behind a hotel connection where speeds are limited.
  • We can use a white or black list to increase test run speeds.
  • We can set custom header values

We created a custom reporter to upload a har file, and then upload it to a project called Har Storage. We actually utilize the same AWS instance we setup PhantomJS on. We now have an easy way to generate some performance trends over time. We setup Page Speed to be run upon upload of the files, so we can actually see the scores we receive for each of our runs. One downside with this project, is it currently only displays the speed — although it is open source, you could add in code to display the results. You could also add in results for YSlow, as it also has a command line option. Below is an example chart generated after loading a page a few times.

Performance Trends over multiple Test RunsYou can also aggregate these into reports depicting various metrics. You can obtain information on the Average, Median, 90th percentile, minimum, and maximum times. This is especially beneficial if you have a multitude of runs over time. Below is an example of the aggregation feature:

Some things we could ultimately do with these combinations:

  • Setup a browser in each of the availability zones and gather metrics for each area of the country / globe
  • Utilize multiple proxies, whereby we limit bandwidth to allow us to run tests of varying speeds to mimic the connection speed of various users.
  • Test builds to ensure our Page Speed score isn’t decreasing between builds
  • Proxy different browsers to test affects of different browsers on page loads.

All in all, it’s been said many times before, if you’re utilizing WebDriver, you should be proxying your tests at this point. You can also run the proxy in embedded mode, don’t push it off because you’re not using AWS or don’t want to work with the REST API!

August 14, 2013 / selenium34

PhantomJS + Plugins + AWS + BrowserMob Proxy

What an adventure to get this working. Long story short, it was RPM hell on a standard aws instance, get the Ubuntu instance from the AWS marketplace.

The other week I needed to do some work testing flash events which are triggered during video playback. Sauce Labs is nice enough to allow us to use browsermob proxy now, but it made little sense to pay them to do this testing. They were basically going to record a video of us watching a video… That’s a good use of those minutes I tell you…

I had a machine in aws which was being utilized as a BrowserMob Proxy instance and a Sauce Connect instance. I decided I would attempt to re-purpose that machine to be a PhantomJS Grid instance which I could connect to. That was easy enough. Our base tests which relied upon the proxy worked just fine. However, the ones which required flash were not working. Makes sense, since flash support was dropped in 1.5. Then I stumbled across this post. I was excited… Then I tried to get it running.

Since this was a default aws instance, I tried the RPM which was listed. It refused to work (I tested with the small script on the blog post). I then tried to use the pre-compiled binary. Enter dependency hell. Things were not going smoothly, I wasn’t having much luck finding all of the dependencies, and they didn’t seem to live in the epel repository, which you can easily enable on a default aws instance. I wasted a day and a half trying to get everything going. I’ll be the first to admit, my Linux chops are a bit rusty, but finding what repo a dependency lives in is a chore. Adding that repo is even worse (my google fu was weak on this, and I wasn’t sure where to find a list of repo rpms or .repo files for download).

I decided to start fresh. I found this image for Ubuntu 12.04 in the AWS marketplace. I got the instance going. Added the dependencies (much easier this time), and then added flash. To get the flash installer, I simply needed to add multiverse to the security.ubuntu.com/ubuntu repo.

After starting up Xvfb and exporting the DISPLAY environment variable, I reran the snap test listed on Ryan’s blog, and lo and behold I had an image of the flash video working. Now I needed it to work as a RemoteWebDriver instance…

Sure enough it was as simple as I hoped it would be. I got my proxy up and running:

browsermob-proxy -port 9090 &
curl -X POST -d 'port=9091' http://localhost:9090/proxy

Then I started the grid:

java -jar selenium-server-standalone-2.35.0.jar -role hub -port 9001 &

Then I started up phantomjs:

phantomjs --webdriver=8080 --webdriver-selenium-grid-hub=http://127.0.0.1:9001 --proxy=127.0.0.1:9091 --load-plugins=yes

Then I used the following as my driver instance:

new RemoteWebDriver(new URL("http://ip.of.aws.instance:9001/wd/hub"), DesiredCapabilities.phantomjs());

To test that flash was working in the grid instance, I wrote a bit of Swing code to grab the screenshot of the remote browser, and display it. Now I can even see what the headless browser is doing while testing.

I can now make calls to the Restful API of browsermob and retrieve the har files where I can verify if my events are taking place as they should.

July 18, 2013 / selenium34

URL Testing

I’m excited for Java 8 having closures. It certainly reduces the amount of code one needs to write, callbacks are as easy as can be, and it’s nice to have other ways to solve a problem. We wished to bring some tests over from our previous project which was using TestNG and data providers. I certainly could have just moved it over, and called it a day, however I felt it was time to look a little bit closer at utilizing closures within groovy.

The first thing we did was create a URLRunner class. It’s goal is to open a connection to the URL input, and then verify that it was able to retrieve the data successfully (for us, this is any code under 300).

class URLRunner implements Runnable {
  def url
  def valid
  
  @Override
  public void run() {
    try {
      url.toURL().openConnection().with {
        connectTimeout = TimeUnit.SECONDS.toMillis(10)
        valid = responseCode < 300
        disconnect()
      }
    } catch(e) {
      valid = false
    }
  }
}

From there we needed a simple way to test these URLs. Again, I could have stuck with what we had, but it was good to tinker. We created a URLExecutorService to handle the threading.

class ULRExecutorService {
  
  def pool = Executors.newFixedThreadPool(30)
  def throwException = true

  def testUrlCollection(collection) {
    /* A lot happens in these lines
       1) we cast the collection to a set to remove duplicates
       2) we strip urls we located which don't begin with http or are blank
       3) we iterate over that collection, creating a new URLRunner for each url we located
       4) we submit the url to the pool
       5) we add the URLRunner to our list (we injected an empty list as our starting value)
    */
    def results = stripNonURLData(collection as Set).inject([]) { list, url ->
      def u = new URLRunner(url:url)
      pool.submit(u)
      list << u
    }
    
    def timeout = 10 * results.size()
    pool.shutdown()
    pool.awaitTermination(timeout, TimeUnit.SECONDS)
    def failed = results.findAll { !it.valid }
    def sb = new StringBuilder("Failed URLS\n")
    failed.each { sb.append(failed.url).append("\n") }
    log.warn(sb)
    if (throwException) {
      assert failed.empty
    }
  }

  def stripNonURLData(collection) {
    collection.findAll { value -> 
      StringUtils.isNotBlank(value) && StringUtils.startsWith(StringUtils.strip(value), "http")
    }
  }
}

You're probably wondering why we have the option to throw exceptions in the Executor.  In certain cases we don't mind if a link is dead -- editors are people, they can make mistakes. We don't want to fail a build if they have a dead link, so when we test all the links on a page, we let those things slide, and simply emit the warning message.

If you're interested, this is what the test for checking the links on a page would look like:

class LinkCheckingTest {

  def executorService = new URLExecutorService(throwException:false)  

  def "test all links on the page"() {
    def doc = Jsoup.connect(baseUrl + "path/to/page.html").get() //We get the baseUrl from the geb configuration, this test simply doesn't run in a browser
    def elements = doc.getElementsByTag("a")
    def urls = []
    elements.each { urls << it.attr("abs:href") } //This should alleviate the check above with links not starting with http
    expect:
    execturorService.testUrlCollection(urls)
  }
}

We have other tests to verify if javascript / css are on the page, where we will fail the build if those files are not located.

 

There you have it, in under 50 lines of code we created a Thread pool, located all the links on a page, opened a straight http connection to the resource(s), and verified they loaded correctly. Granted, it is nearly the same number of lines we had with our data provider / TestNG version, but we do get a huge win. We can easily modify this to use a dynamic ThreadPool size based on environment -- we don't have that luxury with TestNG, it's an annotation value, which needs to be set at compile time. I'm guessing we aren't the only people whose lower environments hate to be pounded with that many threads...

June 21, 2013 / selenium34

How I stopped worrying and learned to love Geb

So as I said in a comment last week I wanted to take a bit of time and give an example of where I thought things were simply easier with Geb. For now I can’t find a live page with this specific element, but I’m sure you can modify this process to something you have seen or are currently using.

So we have a photo gallery we were working on, and it seems to be using some jquery plugin to handle the transitions. It seems to be doing a css transform on the newer browsers, which means that we aren’t changing css classes to indicate which slide is currently active. We were having a bear of a time determining which slide was active. Geb made that quite simple:

    class PhotoGalleryPage extends Page {
      private static final int SLIDE_SIZE = 800

      static content = {
        currentSlideNumber { $("span.current").text() as int }
        previousButton { $("a.prev") }
        nextButton { $("a.next") }
        //The style was simply contained -800 px; or 0 px; or -1600 px; dependent upon which slide was active
        activeSlide { $("div.crsl-slide", style: contains(getOffset() as String)) }
        photo { activeSlide.find("img")
        credit { activeSlide.find("p.credit") }
      }

      def getOffset() {
        (currentSlideNumber - 1) * SLIDE_SIZE
      }
      // Some other helper methods
    }

As you can imagine our specs were quite simple to run after that. Could we have done this in Java? Certainly. Would it have been as small of a footprint? Certainly not.

Some other things which are awesome to have:

  1. Easy access to JavaScript execution
    1. This is fantastic when determining if animations have completed (assuming they are jquery) -- this is all it takes: waitFor (3) { js.exec("return jQuery(":animated").length === 0") }
    2. You could easily do the same for ajax events -- and have a different waiting time, or you could configure one for ajax and have waitFor (ajax) { //code }
  2. Directly access properties on the page
    1. You've drive to a page, just directly access the activeSlide field or the currentSlideNumber
    2. This makes your specs very expressive
  3. More options to find items (you saw I was able to use style: contains above), there are plenty of other options.
    1. Seriously
    2. This is one of the design decisions on selenium that I don't agree with. Had the How enumeration been responsible for creating the locators extending selenium to do custom location would have been so much easier.

Overall I'm quite pleased with Geb.

June 12, 2013 / selenium34

PageObjects

As I’ve sat through many presentations here at the selenium conference this week it is quite apparent where the community is heading, and that’s mobile. This should be of no surprise to you.

What that means is that we need new abstractions for our Page Objects. We likely need many concrete instances of these Page Objects now. One for Android, one for ios, and another for our desktop sites.

The market will have a lot to decide upon. ios-driver 0.7 looks to bring the ability to run on a native device. Appium looks to allow you to run against native and hybrid ios / android applications.

Have you noticed anything exciting in the space which allows for abstracting away the need for multiple page objects?

May 31, 2013 / selenium34

Parallel Running

Well, that sure was a silly mistake. We had overwritten the default gradle test task in our build file. That caused the maxParallelForks property to be ignored… Once we returned back to the default test task we were in business.

Prior to that, we were simply iterating over each of the different drivers in our Geb Configuration in our test step. Now we simply call each of those custom tasks in one line or we run multiple test  steps.

Follow

Get every new post delivered to your Inbox.