Okay, I decided to get the last of the QA tool reviews out on the site so I don’t have to think about them any more. Here it is:
WAPT is a performance/load testing tool developed by SoftLogica. It is a Windows-only tool designed to be very easy to use. A free 30-day trial is available at the company website that allows full functionality with the restriction of 20 virtual users.
- Very easy to record and run tests
- Decent reporting
- Not Free
- No scripting capability
- Doesn’t do well recording AJAX calls
- Standard license - $350 with price per license decreasing with volume
- No distributed load generation
- Unlimited VUs
- WAPT Pro - $700 includes 1 standard license and 2 load agents
- Additional Load Agents - $200
- Provides distributed load generation
Ease of Use
Ease of Installation
WAPT was super easy to install. It installs just as easily as any Windows program. The installer puts everything where it needs to be and you’re set to go.
Setting up a Test
Setting up a test is as simple as starting to record, then going to the sites you want to test. WAPT records all your activity. You then organize it into Initial, Main, and Final actions. The initial actions are executed once per VU per round. These would include things like logging in. The main actions are completed as many times as you tell it to in the virtual user profile. Final actions are completed after all iterations of the main actions.
Once the tests are recorded, you set the Test Volume information and run the test.
I docked WAPT 1 point in this category because the recorder does not do a very good job with AJAX applications. It did okay with Gmail but not so much with LDS Maps.
Running a Test
Running a test is a piece of cake. Once it’s all set up, click the run button and you’re on your way.
Technical Knowledge Needed
Little to no technical ability is needed, since there is no scripting or programming required. You can do most tests just by recording them and playing them back.
WAPT is Windows-only. Requirements:
- Windows 98/Me/2000/XP/Vista (also works on Windows 7 RC)
- Internet Explorer 5.5 or higher
- At least 100 MB free hard drive space (for program + logs)
- Pentium 4 or Athlon XP processor
- 512 MB+ RAM
- Gigabit ethernet
Recording a test
WAPT uses an in-application browser to record test cases. This is a great feature when recording https test cases. Most other tools use a proxy which is unable to detect https GET/POST data.
Unfortunately WAPT doesn’t do a great job recording AJAX application scenarios. Some AJAX requests are missed by the recorder, and thus would have to be created manually. With the Internet moving more and more toward rich applications, this will be a problem if they can’t get it resolved in future versions.
WAPT does not support any kind of scripting.
The reports can be saved as html files which includes some summary data and basic graphs. Results can also be saved to a CSV file if a more detailed data analysis is needed. Figure 1 and Figure 2 below show examples of the information that is available in the HTML reports. As you can see from Figure 2, the data is grouped together by time period. These groupings can be adjusted. The current setting groups them into 10 different segments. That number can be raised or lowered as desired. The list below is all the statistics that are calculated and included in the HTML reports.
- Avg response time (also min/max per page)
- Avg download time (also min/max per page)
- Pages per second
- Total Kbytes sent and received
- Received and sent kbits/sec
- Received and sent kbits/sec/user
- HTTP errors (including user-defined validation errors)
- Socket errors
- Active users
- Total pages, hits, and sessions
WAPT’s graphs are great at showing the trend throughout the test. Each metric is measured for each time period and the graph shows the results live while the test is running. These results are also available in the CSV file exported.
The only external integration WAPT offers is the ability to export to CSV.
In WAPT this is possible, but not as smooth as with jMeter and others. It is done by creating a variable, selecting its function to be “Random from list” or “Ordered list”, and then loading the strings from a file. It this way, each parameter (e.g., street, address, city, etc.) needs its own file. Not the most efficient way. Other programs allow you to use a CSV with each column as its own variable.
Load Generation / Distributed Load Generation
WAPT does not support distributed load testing. WAPT Pro does, but the license for that costs twice what the regular license is. WAPT Pro includes the ability to generate load with 2 machines. More “load agents” as they are called cost $200. There is no limit on VUs, though, so your ability to create load is limited only by your hardware.
WAPT does not include any ability to perform server monitoring. WAPT Pro does.
The documentation that comes with WAPT is great. Because the tool is so simple, it really doesn’t need a lot of documentation. WAPT is not feature-rich, so the little documentation that exists effectively covers all of its functionality.
Fairly pretty UI. Bubbly icons look a little ‘90s, but it’s not bad.
Performance v. Stress Testing
WAPT supports both static-load performance testing and ramp-up stress testing. It also allows you to select whether you want the tests to run a certain number of times or just for a certain time frame. The screenshot below (Figure 3) shows the setup screen where you select how many VUs will be in the test, the ramp-up time (if there is any), and the final load. WAPT also provides an option for “Periodic” load, which alternates between two levels on a schedule you set.
- The built-in report graphs are great, but they use both sides of the y-axis for different scales (Figure 4). There is no indication which line is associated with which scale. You have to turn items off and back on to be able to figure out what goes where.
SoftLogica is based out of Novosibirsk, Russia. Their website appears very out of date, with the most recent press release from 2007, and the newest newsletter from 2006.
Frequency of Releases
New major versions of WAPT are released on average every two years. The newest version (5.0) was released in May, 2007.