kss.core new release
KSS 1.4 released
Refreshing content with KSS
KSS testing how-to for Plone 3.1
Document Actions
Note: This is the print view with all the tutorial pages on one page. The paginated version is available here, if you prefer that.
Environment setup
Buildout creation
Now we download the products we are working on from the development development repository on plone.org.
Check out the existing product skeletton from svn:
svn co https://svn.plone.org/svn/plone/buildouts/plone-coredev/branches/3.1-kss-test 3.1-kss-test
Now you can bootstrap the buildout
python bootstrap.py
Where you can use the absolute path of the python installation you want to use.
Now we are able to launch the buildout creation.
./bin/buildout
Browser setup
Because all tests are created for the english language and we don't need multiple tests for multiple languages we will show how to setup the browser correctly.
If your browser is setup in a different language by english, open the Firefox's preferences and look this example:
And then rearrange the list of languages to have English (US) as the first one (top of the list).
Bootstrapping
The bootstrapping process creates the various objects necessary to run the tests. This process create two objects inside the Zope root: a kss.demo object named demo and a Zuite object named zuite.
To start the bootstrap process just enter the ZMI and then access the view named @@kss-selenium-bootstrap (http://localhost:8080/@@kss-selenium-bootstrap).
When you will see a blank page with "OK" the process is terminated successfully and you can proceed to the rest of tutorial.
After the bootstrapping has been completed, the root of your Zope instance should look like this:
Running the tests
Now we have to go to the Zope root and click on demo (demo)
You have now the KSS demo page where you can run tests, run demos and change the mode on the fly.
You can see that the KSS demo at start is in production mode and you can also switch it to development mode through the "setup" button.
You can see that under the Tests section there are two input boxes and two buttons: the input boxes must be filled with the username and password of your Zope manager account: this is required since some tests need to create content inside your Zope without leaving you logged in until the browser closes. It defaults to admin / admin.
Also, the list under Run Selenium tests shows also all the Applications which have tests registered for: currently, only kss.core (which isn't really an application since it's always there) and Plone.
By clicking the Run button next to the list item you will run the test suites registered for that application.
If this paragraph confused you, just click the Run button near Application Plone: we guarantee it will not make your system explode.
We will see the meaning of Application, registration of tests suites et al later on.
Now you have to change the final part of the path in the address bar to /plone-static-suite.html (for ex. http://localhost:8080/zuite/core/TestRunner.html?test=http://localhost:8080/demo/plone-static-suite.html)
Now you can notice there are three frame in the top of the page. The first at left contains the tests suite that you can run, in the middle you can see the steps of the current test and at the right there is the Selenium TestRunner tool.
In the Selinium TestRunner the first button (1) work to run all tests. The second button (2) work to run the test that you have selected.
Reading the results
Once you get into the Selenium test runner and clicked onto the run all tests button or run the individual test button, you will see the content in the middle and bottom frame change (refer to the picture below to identify the frames correctly).
In the middle frame (2) you can see the commands that the test is
running, while the bottom frame will show the ongoing action with full rendering of the page being tested (3), and the left frame (1) will show the tests that have been completed and their status.
The table that shows the commands is a table that is split in three parts, which replicate the selenium test command/target/value setup (command is the command being run, target is the optional element on which the command is performed, and value is the parameter of the command).
When the command is being processed, it is highlighted in yellow. After a command is executed, it is highlighted in a color that depends on the command outcome.
Commands that have been executed successfully but that were not perfoming any check are highlighted in light green (in the example above, the type command), while commands that have performed a successful check are highlighted in dark green (such as the verifyTextPresent command above).
Commands that have failed, or have performed an unsuccessful check, are shown in red (and we have no example of this, because KSS just works™ :P).
The same color coding holds for the left frame where the executed and yet to be executed tests are shown, with the notable exception that there is no distinction between light and dark green (it wouldn't make much sense anyway).
Making a test (part I-V)
Installing the Selenium IDE
In order to record/make a test you have to install the desktop counterpart of the Selenium test runner, the Selenium IDE.
Actually, the selenium tests that the testrunner runs are nothing but simple HTML files that contains a table in which are specified commands, targets and parameters (it's exactly the table you get shown in the middle frame).
But, to aid you in the process of making a test, OpenQA offers us a simple Firefox extension to record and edit said tests.
Yes, I said record. The basic idea is that you can just switch on recording, click around like your user will do, and the system will record the steps you made and put them into the test HTML file.
The stuff is actually a little more complex than this, but if you are a beginner, I will recommend you to first install Selenium IDE (that you can find at http://selenium-ide.openqa.org/) and watch the nice movie (requires Flash) provided by the Selenium IDE creator.
And then ofcourse, you can play a bit with it and browse through the documentation and very helpful forums they provide.
Tips and tricks
Already here? Wow that was fast. Now, Selenium IDE in record mode works fast and saves you a lot of time, but work mode is not always perfect: it is sometimes necessary to add extra commands.
Here I will give you some simple rules that would make your life easier:
Always check
Always check if an element is actually there before doing anything on it: some commands might fail silently and hence the debug might be a pain after.
Revise the XPath
It is better to always revise the selectors that Selenium generates: this is because sometimes it generates selectors that look like this:
//dl[@id'plone-contentmenu-workflow']/dt/a/span[2]
where something like
//dl[@id'plone-contentmenu-workflow']//a/span[@class='state-private']
would be much better as it doesn't break if someone changes the dt into a dd or moves changes the span order.
Also, if you have an id, it is better to do something like
//$element_tag[@id=$element_id]/$all_the_rest
by keeping the id in the beginning of the xpath selector and then travel down for things that don't have an id.
This is because you can get the id element anywhere it is, and hence you don't get problems if someone moves some portions out of some tags.
Don't use two id selectors in the same xpath, because it's stupid and will make the test break on UI changes.
Wait a second!
Don't use *AndWait if there is no page reloading, because Selenium clearly states that that command will wait for a page reload. Use instead a click and waitFor* command.
Can you see this?
Don't just text just for element existence: assertVisible and waitForVisible are necessary in a lot of cases.
Click on the spans
If, for example, you have some code like this:
<a id="mylink"
href="some/url">
<span>My text</span>
</a>
Don't issue a click command on #mylink but issue it on //a[@id="mylink"]/span. This is because the user will most likely click on the text (so, the span) and not on the a element. Clicking on the a element might masquerade bubbling issues.
I have created my test, what now?
Now that you have learned how to create a Selenium test, record it, and optimize it manually, it's time to save it so the Seleniun testrunner can load it properly.
First, I have to make a brief excursus on how the Selenium tests are organized for KSS.
The first distinction is that each test is first ran in development mode and then into production mode.
Why is that? Because KSS' Javascript, in production mode, gets compressed and concatenated, and said compression might actually make the Javascript not work properly anymore: so a test might run well in development mode, but fail in production mode because the compression screws it.
Secondarily, all the tests are organized into layers.
A layer is simply defined as a set of tests with a set-up routine(s) and tear-down routine(s). This means that some actions that have to be put in place to make the test work (as an example, log in into the site) and that can be done once at the beginning of the tests and then un-done when the tests are finished, are associated to a test layer and all tests in that layer will be logged in when running.
To make it more clear, a test layer does the following:
- Performs set-up actions (ex. log in)
- Runs all the tests
- Performs tear-down actions (ex. log out)
In plone.app.kss we have already some layers in place: how to create new ones or register test layers for your application will be the subject of the next part.
Let's suppose, for now, that you have created a test for a KSS functionality of Plone (let's say, inline editing: but the test is already here, so don't do it). This test requires the user to be able to edit the content, hence it would be desiderable to be logged in as managers.
If you take a look at plone.app.kss/plone/app/kss/demo/selenium_tests you will see three directories:
- run_as_anonymous
- run_as_testmanager
- run_as_testuser
The actual definition of layers happens inside plone.app.kss/plone/app/kss/demo/zopeconfig.py, and you can give a look to this file if you are interested in how to make your own layer: however, the theory behind that will be the subject of the next part.