I love my QA team
The initial idea for creating a QA Toolbar came to me while I was ruing the monotony my good friend and QA Manager Sharon Fowler had to endure on a daily basis. I was concerned with the redundant manual effort she employed while she was reporting bugs: remembering the precise steps she took to produce a bug, accurately expressing the context in which the bug manifested, and then finally, communicating and clarifying that information with the developer(s) responsible for fixing the bug. It seemed wholly unnecessary that multiple humans be responsible for interpreting and re-interpreting data which is readily available at runtime. Also, since the developers are the ones responsible for defining variable names and their values, it seemed logical to include the information available at runtime since it is already presented in developers’ terms. These factors made the QA Toolbar seem like an obvious win for both the QA and Development teams.
Given the obvious benefits, there were only a few problems to solve, and most of them had very simple solutions.
The first problem was retrieving all the data for the QA Toolbar without affecting any existing functionality, or creating extra dependencies in the production environment.
The second problem was injecting HTML into every page which would not affect the layout of the page being tested.
So...How could I make the QA Toolbar appear on each page without affecting production, the page layout, or any other javascript functionality?
The enabling factor was an omni-present JSP include. I added some javascript into the JSP include which would show the QA Toolbar so long as the environment and the client met certain criteria.
Namely, we don’t want the QA Toolbar to show up for you all in production. As a precaution, we also only enable the QA Toolbar for machines inside our office. To address the production issue, I leveraged a variable set by the build and deployment script which indicates whether the application is running in “test mode”. The “test mode” environment variable and an IP filter are used to enable the QA Toolbar in all non-production environments.
Here’s how I injected the QA Toolbar into the page without affecting any other elements:
var qaToolbarDiv = document.getElementById("qaToolbar");
// make sure the qaToolbarDiv exists in the page
if(qaToolbarDiv != null)
{
// set the qa toolbar div as the first element in the body
var body = document.getElementsByTagName("body")[0];
body.insertBefore(qaToolbarDiv, body.childNodes[0]);
}
else
{
// This lets the QA Team know that the tracking header (and thus the QA Toolbar) was not included in the page…we’d better fix this right away!
window.alert('Unable to find QA Toolbar UI Elements. Tracking footer not included on ' + window.location.href);
}
Here is a screenshot of the QA Toolbar when you first load a page (before clicking the tab to expose the QA data):
What information did we need to see?
I figured I’d start with the basics. Which controller was invoked? Which logical view name was chosen? Which JSP is showing (after any filters have been invoked, the JSP may be different than the one originally specified by the controller)? Which user is logged in? What is the user’s email address, user id, and account status? I also figured it’d be nice to have a link directly to that user in our administrative application. Other information proved useful in testing marketing attribution, such as the user’s origin, the placement within that origin, as well as the particular creative the user clicked on. Finally, I added information describing which multivariate tests and which test cells had been selected for the current page view. This wound up being very helpful in debugging multivariate tests. Here’s a screenshot of the QA Toolbar for a basic user:
Yay! I was able to display all this information on the page. I was feeling pretty good after only one day of hacking, but I still hadn’t mitigated the bane of QA’s existence: having to copy and paste information from the page into the bug tracker. The killer feature would be a single click which would create a bug in our issue tracking system, JIRA, and include all this juice in the issue description. Luckily, I was already familiar with the JIRA source code, having extended it to add inline editing of issues as well as some other conveniences. This enabled the implementation of the “Create Jira Issue” button.
Here’s the code which assembles the information from the QA Toolbar for submission to JIRA:
function populateJiraFields()
{
var includePageInformation = document.getElementById("includePageInformation").checked;
var includeUserInformation = document.getElementById("includeUserInformation").checked;
var includeABInformation = document.getElementById("includeABInformation").checked;
var includeTrackingInformation = document.getElementById("includeTrackingInformation").checked;
var includeSessionInformation = document.getElementById("includeSessionInformation").checked;
var description =
'\n\n------------------------------------\n' +
'URL: ' + document.location.href + '\n' +
'Browser: ' + navigator.userAgent + '\n' +
(includeUserInformation ? document.getElementById("userInformation").value : "") +
(includePageInformation ? document.getElementById("pageInformation").value : "") +
(includeABInformation ? document.getElementById("ABInformation").value : "") +
(includeTrackingInformation ? document.getElementById("trackingInformation").value : "") +
(includeSessionInformation ? document.getElementById("sessionInformation").value : "") +
'\n\n---\nPowered by QA Toolbar';
document.getElementById("jiraDescription").value = description;
return true;
}
To see how the runtime information is stored in the HTML page, here’s a glimpse of the rendered source:
<textarea id="userInformation" style="display:none;">
Quick Login: http://www.josh.laddersoffice.com:8080/login?LOGIN_USER=jlevinexxxxxx@theladders.com&LOGIN_PASS=xxxxxxx
User:
Email: jlevinexxxxxx@theladders.com
Password: xxxxxxxx
Subscriber Id: 13xxxxxx
Name: Joshua Levine
Account Status: Basic
Attribution
Basket: dl
Carton: 00
Egg: 01
Subscriptions
Product Name: Basic subscription
Issue Quantity: 0
Ladder: SalesLadder
---
</textarea>
<textarea id="ABInformation" style="display:none;">
AB Selections:
Module xx - Live Chat On/Off Aug06
Cell 3xx- Live Chat On/Off - Test
</textarea>
<textarea id="pageInformation" style="display:none;">
Page:
Web Page ID: 11xxx
Name: SEO Guest Job Search Job view
Controller: com.theladders.web.controllers.xxxxxxxxxxxxxxxxxx
JSP: xxxxxxxxxxxxxx.jsp
View: indexView
</textarea>
<textarea id="trackingInformation" style="display:none;">
User Tracking:
192.168.2.42 - Thu Nov 30 12:25:06 EST 2006 - http://sales.josh.laddersoffice.com:8080/job-search -
192.168.2.42 - Thu Nov 30 12:24:53 EST 2006 - http://sales.josh.laddersoffice.com:8080/Resub -
</textarea>
<textarea id="sessionInformation" style="display:none;">
Session Variables:
- __saved_searches = com.theladders.commons.util.CLASS
- sessionConfig = com.theladders.commons.CLASS
- __profile_stats = com.theladders.commons.util.CLASS
- __preferences = com.theladders.commons.web.CLASS
- qaToolbar = true
- loginRoadblocks = com.theladders.web.controllers.CLASS
- recruiterAuthentication = null
- site = {}
- accesslog.clientAddr = 192.168.2.42
- tracking = com.theladders.commons.web.CLASS
- authentication = User[13xxxxx,jlevinexxxxxx@theladders.com]
- accesslog.user = jlevinexxxxxx@theladders.com
</textarea>
I’m hoping all this inside information will help some of you to better your QA team’s lives forever!
Happy hacking!
- Josh
Special thanks to Joe Ruglio for the QA Toolbar logo and for the infamous QA Toolbar t-shirts.