Extending Selenium 2.0 / WebDriver to support Ajax

Home / Blog / Extending Selenium 2.0 / WebDriver to support Ajax

Recently I started using Selenium 2.0, which uses WebDriver to execute Automated test cases. It works great with normal static, page refreshing applications, but where Ajax is used extensively, this does not works reliably.

Selenium does comes with AjaxElementLocatorFactory, which creates instances of AjaxElementLocator. Now the idea is, if you send an Ajax request, this ElementLocator waits for 250 milliseconds to look for the element, till it ultimately times out (configurable). The only exposed API from Selenium, that I found, was PageFactory, whose main purpose is to create a DefaultElementLocatorFactory, which does not wait.

So here are some of the modifications I had to make (using trial and error), to get this framework to work reliably (if you had similar luck, join the bandwagon :)).

First modification – Use what is provided. To expose AjaxElementLocatorFactory, I had to create a new extension of PageFactory class. Here it is -

  
        package com.brim.selenium.locator;
        import java.lang.reflect.InvocationTargetException;
		import java.lang.reflect.Method;

		import org.apache.commons.lang.StringUtils;
		import org.openqa.selenium.WebDriver;
		import org.openqa.selenium.support.PageFactory;
		import org.openqa.selenium.support.pagefactory.AjaxElementLocatorFactory;
		import org.openqa.selenium.support.pagefactory.ElementLocatorFactory;

		import com.brim.selenium.util.SeleniumUtility;

		public class AjaxEnabledPageFactory extends PageFactory {
		    // instantiatePageMethod is defined private in PageFactory, so use reflection to expose it,
			// or copy and paste. I hate copy/paste so implemented it using reflection
			
			static Method instantiatePageMethod = null;
			static {
				Method[] methods = PageFactory.class.getDeclaredMethods();
				for (Method method : methods) {
					if (StringUtils.equals(method.getName(), "instantiatePage")) {
						instantiatePageMethod = method;
						instantiatePageMethod.setAccessible(true);
					}
				}
			}
            
            // these are defined static in PageFactory, so had to copy this methods, to call my instantiatePage method
            // unfortunately, Java does not allow over-riding static methods, though compiler does not complain :( 			
			public static  T initElements(WebDriver driver, Class pageClassToProxy) {
				T page = instantiatePage(driver, pageClassToProxy);
				initElements(driver, page);
				return page;
			}
             
			 // here is where i create AjaxElementLocatorFactory instance
			 // SeleniumUtility.timeOutInSeconds parameter is my own static parameter which is read from a property file
			 // more about it later
			public static void initElements(WebDriver driver, Object page) {
				final WebDriver driverRef = driver;
				initElements(new AjaxElementLocatorFactory(driverRef, SeleniumUtility.timeOutInSeconds), page);
			}
             
			 // another interesting issue i found, that sporadically Selenium/Webdriver throws
			 // StaleElementException, so it finds it, and while editing it throws that its stale,
			 // StaleReferenceAwareFieldDecorator does just that, if it sees the element is stale, look again. 
			 // More about it later.
			public static void initElements(ElementLocatorFactory factory, Object page) {
				final ElementLocatorFactory factoryRef = factory;
				initElements(new StaleReferenceAwareFieldDecorator(factoryRef), page);
			}
            
			 // and here is the call to PageFactory's private instantiatePage method using my new AjaxEnabledPageFactory class.
			protected static  T instantiatePage(WebDriver driver,
					Class pageClassToProxy) {
				try {
					return (T) instantiatePageMethod.invoke(
							AjaxEnabledPageFactory.class, driver, pageClassToProxy);
				} catch (IllegalArgumentException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				} catch (IllegalAccessException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				} catch (InvocationTargetException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
				return null;
			}
		}

  
 

So, now the first modification is done, how to use it-

    
	  PageClass pageObject = AjaxEnabledPageFactory
				.initElements(driver, PageClass.class);
	
  

Now, when I run my testcases, with configurable timeOut of 5 seconds, Selenium tries every 250 milliseconds to look the element, for max of 5 seconds and then throws ElementNotFoundException. Cool, everything should work now, I thought so too, then sporadically I will get StaleElementReferenceException. Now, PageFactory uses DefaultFieldDecorator, which decorates every WebElement / RenderedWebElement instance with a proxy. Any time, you execute a method on a WebElement, proxy forwards the request to the locator to locate the element first and then execute the method. But, sometimes, locator locates the webElement, but just before it tries to type in the textbox or click the button, the element is removed from DOM. So much for Ajax :). Simple solution, capture the StaleElementReferenceException, look again using locator and execute the method. Here is how to do it.

      
	    import java.lang.reflect.InvocationHandler;
		import java.lang.reflect.InvocationTargetException;
		import java.lang.reflect.Method;
		import java.lang.reflect.Proxy;

		import org.apache.log4j.Logger;
		import org.openqa.selenium.RenderedWebElement;
		import org.openqa.selenium.StaleElementReferenceException;
		import org.openqa.selenium.WebElement;
		import org.openqa.selenium.internal.WrapsElement;
		import org.openqa.selenium.support.pagefactory.DefaultFieldDecorator;
		import org.openqa.selenium.support.pagefactory.ElementLocator;
		import org.openqa.selenium.support.pagefactory.ElementLocatorFactory;
		import org.openqa.selenium.support.pagefactory.internal.LocatingElementHandler;
	   
	    public class StaleReferenceAwareFieldDecorator extends DefaultFieldDecorator {
			private static final Logger logger = Logger
					.getLogger(StaleReferenceAwareFieldDecorator.class.getName());

			public StaleReferenceAwareFieldDecorator(ElementLocatorFactory factory) {
				super(factory);
			}

			protected WebElement proxyForLocator(ClassLoader loader,
					ElementLocator locator, boolean renderedProxy) {
				InvocationHandler handler = new StaleReferenceAwareElementLocator(
						locator);

				WebElement proxy;
				if (renderedProxy) {
					proxy = (RenderedWebElement) Proxy
							.newProxyInstance(loader, new Class[] {
									RenderedWebElement.class, WrapsElement.class },
									handler);
				} else {
					proxy = (WebElement) Proxy.newProxyInstance(loader, new Class[] {
							WebElement.class, WrapsElement.class }, handler);
				}
				return proxy;
			}

			private static class StaleReferenceAwareElementLocator extends
					LocatingElementHandler {
				private final ElementLocator locator;

				public StaleReferenceAwareElementLocator(ElementLocator locator) {
					super(locator);
					this.locator = locator;
				}
                
				// here is where the magic happens. For a configurable number of times (I configured
				// 5 times, locater finds the element, and then tries to invoke the method on the element
				// In case StaleElementReferenceException is thrown, try again.
				public Object invoke(Object object, Method method, Object[] objects)
						throws Throwable {
					int count = 0;
					WebElement element = null;
					while (count < SeleniumUtility.numberOfTries) {
						try{
							element = locator.findElement();
						}catch(Exception ex){
							//logger.debug("Element not found", ex);
							SeleniumUtility.waitFor(250);
							count++;
							continue;
						}
						if ("getWrappedElement".equals(method.getName())) {
							return element;
						}

						try {
							return invokeMethod(method, element, objects);
						} catch (StaleElementReferenceException ex) {
							//logger.debug("Error locating element", ex);
						}
						count++;
					}
					throw new RuntimeException("Cannot invoke " + method.getName()
							+ " on element " + SeleniumUtility.getElementName(element)
							+ ". Cannot find it");
				}

				private Object invokeMethod(Method method, WebElement element,
						Object[] objects) throws Throwable {
					logger.debug("Invoking " + method.getName() + " on "
							+ SeleniumUtility.getElementName(element));
					try {
						return method.invoke(element, objects);
					} catch (InvocationTargetException e) {
						// Unwrap the underlying exception
						throw e.getCause();
					} catch (IllegalArgumentException e) {
						// Unwrap the underlying exception
						throw e.getCause();
					} catch (IllegalAccessException e) {
						// Unwrap the underlying exception
						throw e.getCause();
					}
				}
			}
		}
      
   

Using these extensions, I atleast got rid of Ajax related issues. There were more modifications to extend the WebDriver’s FindBy annotations to include dynamic pages, where a link when clicked adds a new row in a table, and the id/name differs just by the index.
I will cover it in next article. Btw, I used Selenium2.0a7 release.

Showing 7 comments
  • Ale

    ElementLocatorFactory finder = new AjaxElementLocatorFactory(driver,
    DRIVER_WAIT); //DRIVER_WAIT = 30 seconds
    PageFactory.initElements(finder, this);

  • yagish sharma

    Ale, I saw this example on code.google wiki, and tried it. It does work. This code could be easily encapsulated in a factory method, instead of repeating in every other test suite. In fact, to use custom StaleReferenceAwareFieldDecorator, I can easily write -

    public void initializePageElements(WebDriver driver, Class pageClass){
    ElementLocatorFactory finder = new AjaxElementLocatorFactory(driver, timeout);
    StaleReferenceAwareFieldDecorator decorator = new StaleReferenceAwareFieldDecorator(finder);
    PageFactory.initElements(decorator, pageClass);
    }

    I agree, a factory method will cut down code.

  • yagish Sharma

    Here is the refactored class -


    public class AjaxEnabledPageFactory {

    @SuppressWarnings("unchecked")
    public static T initializePage(WebDriver driver, Class pageClass) {
    Object page = createInstance(driver, pageClass);
    PageFactory.initElements(new StaleReferenceAwareFieldDecorator(
    new AjaxElementLocatorFactory(driver,
    SeleniumUtility.timeOutInSeconds)), page);
    return (T) page;
    }

    public static T createInstance(WebDriver driver, Class pageClassToProxy){
    try {
    try {
    Constructor constructor = pageClassToProxy.getConstructor(WebDriver.class);
    return constructor.newInstance(driver);
    } catch (NoSuchMethodException e) {
    return pageClassToProxy.newInstance();
    }
    } catch (InstantiationException e) {
    throw new RuntimeException(e);
    } catch (IllegalAccessException e) {
    throw new RuntimeException(e);
    } catch (InvocationTargetException e) {
    throw new RuntimeException(e);
    }
    }
    }

    Here is the new usage pattern –


    Object pageObject = AjaxEnabledPageFactory
    .initializePage(driver, pageclass);

  • Shannon Code

    Thanks for this, I have been trying to create my own By methods for a while. This post helped me greatly.
    -Shannon Code

  • Dan Tran

    Does this code work with selenium 2.x and where can get get SeleniumUtil class?

    -D

  • Dan Tran

    Does this still work with Selenium 2.x and where can get a hold of com.brim.selenium.util.SeleniumUtility source?

    Thanks for the great post

    -D

  • yagish sharma

    Dan,

    I have not tested it with the latest Selenium 2.x version, but yeah, it works for Selenium 2.0 version. Here is the SeleniumUtility class, I am afraid this includes several changes. I think I removed the static fields for numOfTries, which are set to 5, but have just helper methods, which were tweaked to my comfort.


    public class SeleniumUtility {

    /***
    * Gets an element by name or id
    *
    * @param driver
    * @param nameOrID
    * @return
    */
    public static WebElement getByNameOrID(WebDriver driver, String nameOrID) {
    WebElement element = null;
    // try getting the element by name
    try {
    element = driver.findElement(By.name(nameOrID));
    } catch (Exception ex) {

    }
    if (null == element) {
    try {
    element = driver.findElement(By.id(nameOrID));
    } catch (Exception ex) {

    }
    }
    return element;
    }

    /***
    * Gets select option by xpath
    *
    * @param driver
    * @param nameOrID
    * @param optionXPath
    * @return
    */
    public static WebElement getSelectOptionByXPath(WebDriver driver,
    String nameOrID, String optionXPath) {
    WebElement selectElement = getByNameOrID(driver, nameOrID);
    WebElement options = selectElement.findElement(By.tagName("option"));
    return selectElement.findElement(By.xpath(optionXPath));
    }

    /**
    * Gets select option by option value
    *
    * @param driver
    * @param nameOrID
    * @param optionValue
    * @return
    */
    public static WebElement getSelectOptionByValue(WebDriver driver,
    String nameOrID, String optionValue) {
    String optionXPath = "//option[@value='" + optionValue + "']";
    return getSelectOptionByXPath(driver, nameOrID, optionXPath);
    }

    public static WebElement getSelectOptionByValue(WebDriver driver,
    WebElement selectElement, String optionValue) {
    return selectElement.findElement(By.xpath("//option[@value='"
    + optionValue + "']"));
    }

    public static boolean assertNull(WebElement element) {
    try {
    // call any method on the element
    element.isEnabled();
    } catch (Exception ex) {
    return true;
    }
    return false;
    }

    public static boolean assertNotNull(WebElement element) {
    try {
    // call any method on the element
    element.isEnabled();
    } catch (Exception ex) {
    return false;
    }
    return true;
    }

    /***
    * Gets input type by type
    *
    * @param driver
    * @param type
    * @return
    */
    public static WebElement getInputItemByType(WebDriver driver, String type) {
    String xpath = "//input[@type='" + type + "']";
    return driver.findElement(By.xpath(xpath));
    }

    public static WebElement getRadioWithIndex(WebDriver driver,
    String nameOrId, String index) {
    // String xpath =
    // "//input[@id='"+nameOrId+"' and @type='radio']["+index.trim()+"]";
    String xpath = "//input[@id='" + nameOrId + "' and @type='radio'][1]";
    WebElement radio = null;
    try {
    radio = driver.findElement(By.xpath(xpath));
    } catch (Exception ex) {

    }
    // try with name now
    // xpath =
    // "//input[@name='"+nameOrId+"' and @type='radio']["+index.trim()+"]";
    xpath = "//input[@name='" + nameOrId + "' and @type='radio']/[2]";
    try {
    radio = driver.findElement(By.xpath(xpath));
    } catch (Exception ex) {

    }
    return radio;
    }

    /***
    * Gets input type by value
    *
    * @param driver
    * @param type
    * @return
    */
    public static WebElement getInputItemByValue(WebDriver driver, String value) {
    String xpath = "//input[@value='" + value + "']";
    return driver.findElement(By.xpath(xpath));
    }

    /***
    * Gets input type by type and value
    *
    * @param driver
    * @param type
    * @param value
    * @return
    */
    public static WebElement getInputItemByTypeAndValue(WebDriver driver,
    String type, String value) {
    String xpath = "//input[@type='" + type + "' and @value='" + value
    + "']";
    return driver.findElement(By.xpath(xpath));
    }

    public static WebElement getInputItemByTypeAndName(WebDriver driver,
    String type, String name) {
    String xpath = "//input[@type='" + type + "' and @name='" + name + "']";
    return driver.findElement(By.xpath(xpath));
    }

    /***
    * Gets image by alt text
    *
    * @param driver
    * @param type
    * @return
    */
    public static WebElement getImageByAltText(WebDriver driver, String altText) {
    String xpath = "//img[@alt='" + altText + "']";
    return driver.findElement(By.xpath(xpath));
    }

    public static WebElement getLinkByNameOrID(WebDriver driver, String nameOrID) {
    String xpath = "//a[@id='" + nameOrID + "']";
    WebElement link = null;
    try {
    link = getElementByXPath(driver, xpath);
    } catch (Exception ex) {

    }
    if (null != link) {
    return link;
    }
    xpath = "//a[@name='" + nameOrID + "']";
    try {
    link = getElementByXPath(driver, xpath);
    } catch (Exception ex) {

    }
    return link;
    }

    /***
    * Gets element by xpath
    *
    * @param driver
    * @param xPath
    * @return
    */
    public static WebElement getElementByXPath(WebDriver driver, String xPath) {
    return driver.findElement(By.xpath(xPath));
    }

    private static void waitForASecond() {
    try {
    Thread.sleep(1000);
    } catch (Exception ex) {
    // do nothing
    }
    }

    /***
    * Waits the till element is displayed
    *
    * @param driver
    * @param elementNameOrID
    */
    public static WebElement waitForElement(WebDriver driver,
    String elementNameOrID) {
    WebElement webElement = null;
    while (null == webElement) {
    webElement = getByNameOrID(driver, elementNameOrID);
    if (null != webElement)
    break;
    waitForASecond();
    }
    return webElement;
    }

    public static WebElement waitForSelectElement(WebDriver driver,
    String elementNameOrID, String optionValue) {
    WebElement webElement = null;
    while (null == webElement) {
    webElement = getSelectOptionByValue(driver, elementNameOrID,
    optionValue);
    if (null != webElement)
    break;
    waitForASecond();
    }
    return webElement;
    }

    public static WebElement waitForEnabled(WebDriver driver,
    String elementNameOrID) {
    WebElement webElement = null;
    while (null == webElement || !webElement.isEnabled()) {
    webElement = getByNameOrID(driver, elementNameOrID);
    if (null != webElement && webElement.isEnabled())
    break;
    waitForASecond();
    }
    return webElement;
    }

    public static WebElement waitForEnabled(WebElement element){
    while(! element.isEnabled()){
    waitForASecond();
    }
    return element;
    }
    public static WebElement waitForInputElementByTypeAndValue(
    WebDriver driver, String type, String value) {
    WebElement webElement = null;

    while (null == webElement) {
    try {
    webElement = getInputItemByTypeAndValue(driver, type, value);
    } catch (Exception ex) {

    }
    if (null != webElement)
    break;
    waitForASecond();
    }
    return webElement;
    }

    /***
    * Waits till the title is displayed
    *
    * @param driver
    * @param title
    */
    public static void waitForTitle(WebDriver driver, String title) {
    String pageTitle = null;
    while (StringUtils.isBlank(pageTitle)
    || !StringUtils.equals(pageTitle, title)) {
    try {
    pageTitle = driver.getTitle();
    } catch (Exception ex) {

    }
    if (StringUtils.isNotBlank(pageTitle)
    && StringUtils.equals(pageTitle, title))
    break;
    waitForASecond();
    }
    }

    public static WebElement waitForElementSelected(WebElement webElement) {
    if (null == webElement) {
    return webElement;
    }
    while (!webElement.isSelected()) {
    waitForASecond();
    }
    return webElement;
    }
    }

Leave a Comment