JMeter is a free and open source tool created by the Apache Software Foundation to carry out performance testing. The tool is a Java-based application and was originally designed to test web applications.
It allows for load simulations to be carried out through different types of applications and protocols:
● Web – HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET, etc.)
● SOAP/REST web services
● FTP, FTPS
● Database via JDBC
● LDAP
● Message-oriented middleware (MOM) via JMS
● Mail – SMTP, POP3 e IMAP
● Native commands or shell scripts
● TCP
● Java objects
JMeter features an integrated development environment (IDE), which allows for tests to be created quickly using its script-recording function from a web browser and to be subsequently executed.
It includes a command line execution mode, which makes it possible to run the tests from any Java-compatible operating system (Windows, Linux, Mac OS, among others).
It provides the means to export the test results through a complete HTML report.
JMeter allows the extension of its functionalities through add-ons. Several are available through the Apache community, but it is also possible to build new ones when needed.
It also allows an integration with tools and libraries in continuous integration schemes, such as Maven, Gradle and Jenkins.
Understanding that JMeter is not a web browser is essential; the tool works on a protocol level. More specifically, JMeter does not execute embedded code on the pages as a web browser would; therefore, it does not execute JavaScript and does not render HTML pages.
A test plan in JMeter defines a tree structure of how, when and what to test, providing for the execution of a sequence of actions.
The element categories that JMeter handles are:
- Requests (Sampler): the different types of requests for different protocols can be found here.
- Logic Controllers: the logic controllers that allow you to define the flow of the tests are located here.
- Pre Processors: this category comprises all the elements that can be executed before making a request.
- Post Processors: this category comprises all the elements that can be executed after making a request.
- Assertions: all the elements used to carry out verifications and validations during the execution of the tests are grouped here.
- Timer: the elements related to waiting times are located here.
- Script fragments (Test Fragment): this category contains a single element with the same name. This element allows us to define script fragments to be reused in different sections of the script.
- Config Element: this category contains all the configuration elements for the script.
- Listener: the elements related to the results listeners and test execution reports can be found here.
The threads or users are represented in JMeter through groups of threads.
Each group of threads represents a set of system users, and they are used to execute the performance tests to simulate the concurrence of users on the application.
This is the very first options of Thread Group element, but not many people pay attention to it. It determines what happens if a sampler error occurs, either because the sample itself failed or an assertion failed. The possible choices are:
Continue (default)- ignore the error and continue with the test and run next sampler.
Start Next Loop – ignore the error, start next loop and continue with the test
Stop Thread – current thread exits
Stop Test – the entire test is stopped at the end of any current samples. It means the pending samplers are still run until done.
Stop Test Now – the entire test is stopped immediately. Any current samplers are interrupted if possible.
The ramp-up period tells JMeter how long to take to “ramp-up” to the full number of threads chosen.If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun.
The delayed threads creation functionality was introduced in JMeter version 2.8. “Delayed threads” feature that allows allocating memory for new threads only when they start their execution. Previously, when you were going to spin up 1000 users during your performance tests, even if you had a ramp up time for users, JMeter allocated memory for all the threads right away.
This mode enables rapid validation of a Thread Group by running it with 1 thread, 1 iteration, no timers and no Startup delay set to 0.
The 3 first properties can be modified by setting in jmeter.properties:
testplan_validation.nb_threads_per_thread_group : Number of threads to use to validate a Thread Group, by default 1
testplan_validation.ignore_timers : Ignore timers when validating the thread group of plan, by default true.
testplan_validation.number_iterations : Number of iterations to use to validate a Thread Group, by default 1.
There are at least four ways to run JMeter behind the Proxy.
1. –> Config the Proxy Server into each HTTP Request.
In HTTP Request Sampler, you can find the Proxy Server under Advanced Tab.
2. Config the Proxy Server into HTTP Request Defaults.
3. Launch JMeter from the command line with the following parameters:
jmeter -H localhost -P 8888 -u username -a password -N localhost
NOTE: You can also use –proxyHost, –proxyPort, –username, and –password as parameter names.
jmeter –proxyHost localhost –proxyPort 8888
jmeter -n -H localhost -P 8888 -t C:\jmeter\my-test-plan.jmx -l C:\jmeter\my-test-plan-result.jtl -j C:\jmeter\my-test-plan-result.log
4. Set the Proxy properties into the System properties file.
Open the system.properties in edit mode, this file is located under /JMeter/bin/ directory
Add the following properties to the end of file.
If a non-proxy host list is provided, then JMeter sets the following System properties:
http.proxyHost=localhost
http.proxyPort=8888
https.proxyHost=localhost
https.proxyPort=8888
http.nonProxyHosts=example.com
https.nonProxyHosts=example.com
– setUp Thread Group: will be executed BEFORE the test proceeds to the executing of regular Thread Groups.
– tearDown Thread Group: will be executed AFTER the test proceeds to the executing of regular Thread Groups.
Deleting users that were created in the setUp Thread Group.
Cleaning up the system, deleting the data which were created during the test.
Send the email or any kind of notification to notify that the test has been stopped.
Collect some reporting stuff on response data.
A group of virtual users is represented in the JMeter application by a Thread Group.
A thread group is the base element (parent) needed in every test plan.
It is actually an assembly of threads executing the same scenario.
Thread Group – This can be used for simple load test scenarios, where you set a predefined number of users and ramp-up period and you start running your script in times. We can use Thread Group for Stress Testing also by increasing the concurrent users. This can be used for Endurance Testing by maintaining the load for predefined period of time.
JMeter Arrivals Thread Group element is specially created for the case where no. of orders to be submitted in an hour; is known. In such type of scenarios, more preference is given to the quantity of order irrespective no. of users. Apache JMeter adjust the number of threads according to the target arrival rate.
“Arrivals Rate” word is used for the rate of iteration (order submission). If I say 10 licenses are purchased in a minute which signifies that 10 iterations need to be started and completed in 1 minute. Hence the arrival rate will be 10/min. If I say 1000 movie tickets booking in an hour (16.6 = ~17/min), then it signifies that around 17 iterations should be started and completed in 1 minute, so arrival rate will be 17.
Target Rate = 120 Arrivals / min
Ramp-up Time: 1min
Step-Count: 10
Hold: 2min
Total Arrivals = Arrivals between 0-30 Sec + Arrivals between 30-60 Sec + Arrivals between 60-180Sec
0-30 Sec = 30 Arrivals
30-60 Sec = 60 based on (120 * 30/60)
60-180 = 120*2 = 240
Total Arrivals = 30+60+240 = 330 Arrivals/Min
It is similar to arrivals thread group, the difference is that this thread group does not have ramp up time and steps options.
The Start and End Values: Start value means arrival rate at the beginning of the time frame (duration) and end value means arrival rate at the end of the time frame (duration).
Start Value=30
End Value=60
Duration= 60 Sec
30 Arrivals will be added per Sec at the beginning of timeframe. 1 arrival for every 2 sec to reach target 60 arrivals by end of 60 sec.
Adding up all the arrival rates from 1st to 60th second gives total number of arrivals=30+30+31+31+32+32+….+60 ~= 2700.
This Thread group is also suitable for goal oriented scenarios, but the goal here is to have control over the number of concurrent users over a period of time.
Target Concurrency: The number of concurrent users which should be maintained after the ramp up.
Ramp Up Time: Timeframe required to reach target concurrency rate.
Ramp Up Steps: It refers to the granularity of the concurrency increase rate. More steps results in smoother pattern.
Hold Target Rate Time: The duration to maintain the target concurrency before starting to gradually shut down all threads.
Thread Iterations Limit: Limits the number of iterations.
The ultimate thread group is highly customizable and unlike the arrivals, or freeform and concurrency thread groups, this one kills the active threads after the set time has expired.
The start threads count field: You set the target thread concurrency.
The initial delay: Set how much time you wait before starting to execute the threads from that specific row.
The startup time: The ramp up time for all the threads set in the thread count. For example, for 20 threads and a start time of 20 seconds, you would have 1 new thread firing up every 1 second.
The hold load for field: This is where you set the duration to maintain the defined number of concurrent users and is totally independent from the ramp-up time.
The shutdown time: The time interval for killing all the active threads.
Timers can help to simulate a virtual user’s “think time”.
Timers allows JMeter to delay/pause between each request which a thread makes.
Rule 1: The Timers are executed before the sampler’s execution but after the PreProcessor.
Rule 2: The response time of the sampler does not include the execution time of the Timer.
The purpose of Constant Timer is to pause for the same amount of time between requests from each thread.
Parameterized the delay time: ${timer}
Random the delay time: ${__Random(1000,5000,)}
Uniform random timer is one of the Jmeter timer which is used to generate fixed+random amount of time delay between 2 requests.
Total amount of delay = Random Delay Maximum + Constant Delay Offset
Random Delay Maximum : This is outer boundary limit of random time delay. It will decide maximum random pause time.
Constant Delay Offset : This is the fixed delay time which will be added in random delay.
For Example:
Random Delay Maximum = 3000 milliseconds and Constant Delay Offset = 7000 milliseconds. That means it will put random delay between 7 seconds to 10 seconds between two requests.
You can use Synchronizing Timer If you wants to release X number of threads at given point. It will block the threads until given X number of threads have been reached at specific point. It will release all threads at once when given number of threads are reached on specific point of time.
You can use synchronizing timer on controller level or request level as per your requirement.
Using constant throughput timer, You can decide how many samples should be executed per minute. Constant throughput timer will add random pauses between requests during test execution to match required throughput.
Note : If you are using constant throughput timer in your software load test plan and server is not capable to handle the load or any other time consuming elements are available in your test plan then your targeted throughput(Which is set in constant throughput timer) will be not achieved.
Test Fragment:
Test Fragment element is a special controller which can be added directly under JMeter test plan like Thread Group. But It does nothing except holding other elements inside!! It gets executed only when it is referenced by a Module/Include controller from other Thread Groups. It acts like a library of reusable of scripts.
Module Controller:
Module Controller in JMeter is useful in referencing any of the logic controller in the JMeter Test Plan.
Parameterized Controller:
Parameterized Controllers contains the ‘User Defined Variables’ section, where you can specify your parameters. Put the credentials of the first user in the first parameterized controller and the second user credentials in the second parameterized controller.
Include Controller:
As a Module controller is used to call a logic controller in the Test Plan, Include Controller is used to reference an existing .jmx file itself.
Logic Controllers help you to control the flow of the order of processing of samplers in a thread. It can also change the order of requests coming from their child elements.
| Interleave Controller | Interleave controller will select only one sample / request stored in it, to run in each loop of the thread. It will execute the samplers in sequentially. |
| Once Only Controller | Requests added to this controller will run only once irrespective of number of users/loop count specified in the thread group. This comes into use, when one wants all other requests run more than once but a particular request or set of requests to run only once during the entire test plan. |
| Include Controller | Include Controller is made to use an external test plan. This controller allows the usage of multiple test plans in JMeter. |
| Runtime Controller | Runtime Controller controls the execution of its samplers/requests for the given time. For example, if you have specified Runtime(seconds) to 20, JMeter will run your test for 20 seconds only. |
| Loop Controller | Loop Controller will run the samplers/requests stored in it for the definite number of times or forever (if forever checkbox is selected). |
| Module Controller | The module controller adds modularity to the JMeter Test Plan. Module controller allows you to redirect test execution to a given fragment of the test or you can say selected module. |
| Recording Controller | Recording Controller is a place holder where the proxy servers can save recorded requests. It has no effects on test execution. |
| Transaction Controller | Useful to measure over all time taken to perform complete transaction. Transaction controller will add additional sample after nested samples to show total time taken by it’s nested samples. Generate parent sample Transaction controller has option “Generate parent sample” and “Include duration of timer and pre-post processors in generated sample” |
| Random Order Controller Example | Random order controller in apache jmeter is used to execute requests of jmeter software load test plan in random order. This is the only difference between simple controller and random order controller. |
| Random Controller | As you know, controllers in jmeter are useful to control execution flow in different ways. Random controller will execute any request randomly. It will pick any request from it’s request tree and force to execute is. Every time it will pick different request randomly. |
| Switch Controller | As name suggest, You can switch your targeted software application page request to run using Switch Controller. |
| IF Controller | If controller in jmeter allows you to set condition to evaluate it and based on condition evaluation result it will decide to run or not to run if controller’s child elements. If condition evaluation result is positive then child elements of if controller will run else child elements of if controller will not run. “${title}”==”Yahoo” |
| Throughput Controller | This controller does not control the Throughput. Primary objective of Throughput Controller is to define and control the User Load for each group separately under same Thread Group. Throughput Controller has 2 options: Total Executions, Percent Executions. 1) Total Executions: All child requests placed under this controller will be executed as per defined number. The thread creation will stop after the defined number of execution have occurred. – Per User Checked: The number given for throughput will be the total number of executions for current controller. – Per User Unchecked: Total number of Threads(Users) given under Thread Group will be the total number of executions for current controller. 2) Percent Executions: All child requests placed under this controller will be executed according to the percentage defined under Throughput field. |
| While Controller | The purpose of the While Controller is to repeat a given set of actions until the condition is broken. For example, you simulate a user who buys all the goods within a random product category. The number of goods to buy varies depending on the category. In JMeter, the While controller runs all the Samplers which are located underneath, unless some predetermined “Condition” is “true” where “Condition” could be: blank: exits when the last sampler in the loop fails, LAST: the same as blank, but with an additional check for the last Sampler before the loop result. If the preceding Sampler is not successful, the While Controller and its children won’t be executed at all, JMeter Function or Variable: children will be run until a variable or function evaluation result is false, JMeter Property: it works the same way as a Function or Variable, but assumes the value of a JMeter property instead. “${myVar}” != “some value” – doesn’t work, as it is neither a function nor a single variable. ${__javaScript(“${myVar}” != “some value”,)} – works OK as the expression resolves it to either be “true” or “false”. ${__javaScript(${counter}<=5)} ${__javaScript(parseInt(vars.get(“counter”))<=5)} ${__javaScript(! “${CFY_g4}”.endsWith(‘label1’) || (! “${CFY_g6}”.match(‘Current Fiscal YTD’)) )} |
Configuration elements can be used to set up defaults and variables for later use by samplers.
Types of Config Elements:
CSV Data Set Config – CSV Data Set Config is used to read lines from a file, and split them into variables. We can use __CSVRead() and _StringFromFile() JMeter in built functions to read data from file.
By default, the file is only opened once, and each thread will use a different line from the file. However the order in which lines are passed to threads depends on the order in which they execute, which may vary between iterations. Lines are read at the start of each test iteration.
| FTP Request Defaults | FTP Request Defaults config element is associated with FTP Request Sampler. This element is used when your going to send multiple requests to same FTP server. |
| HTTP Authorization Manager | The Authorization Manager lets you specify one or more user logins for web pages that are restricted using server authentication. You see this type of authentication when you use your browser to access a restricted page, and your browser displays a login dialog box. JMeter transmits the login information when it encounters this type of page. |
| HTTP Cookie Manager in Jmeter | Each JMeter thread has its own cookie storage area. The use of cookies eventually becomes necessary when your application has to maintain a session. Once you login to an application, it maintains the session of that user so that he/she can work inside the application. If that session is not maintained (via Cookie Manager in case of JMeter), then the user will be logged out of the application as soon as he/she sends the next request, which requires Authentication. You can take it like this, if your application has a session or uses cookies, then your script will not work without adding cookies in cookie manager, as then your script will not be able to maintain the session and the users (Threads) will be kicked-off the application as soon as they enter into it. Each JMeter thread has its cookie storage area. So, if you are testing a web site that uses a cookie for storing session information, each JMeter thread will have its session. |
| HTTP Request Defaults | This element lets you set default values that your HTTP Request controllers use. For example, if you are creating a Test Plan with 25 HTTP Request controllers and all of the requests are being sent to the same server, you could add a single HTTP Request Defaults element with the “Server Name or IP” field filled in. Then, when you add the 25 HTTP Request controllers, leave the “Server Name or IP” field empty. The controllers will inherit this field value from the HTTP Request Defaults element. |
| HTTP Header Manager | The Header Manager lets you add or override HTTP request headers. Ex: User-Agent Accept Accept-Encoding Accept-Language |
| User Defined Variables | The User Defined Variables element lets you define an initial set of variables , just as in the Test Plan . Note that all the UDV elements in a test plan – no matter where they are – are processed at the start. UDVs can be used with functions such as __P(), for example: HOST ${__P(host,localhost)} |
| Counter | Allows the user to create a counter that can be referenced anywhere in the Thread Group. The counter config lets the user configure a starting point, a maximum, and the increment. The counter will loop from the start to the max, and then start over with the start, continuing on like that until the test is ended. |
| Login Config Element | The Login Config Element lets you add or override username and password settings in samplers. |
| Simple Config Element | ‘Simple Config Element’ is used to add or override arbitrary values in samplers. You can choose the name of the value and the value itself. |
| JAVA Request Defaults | ‘Java Request Defaults’ is used to set default parameters to pass them into Java Request. Sleep_time Sleep_mask Label Response Code ResponseMessage Status SamplerData ResultData |
When testing your APIs, web service or other system parts, you might need to record or retrieve data from a database.
Before you start working with a database by using JMeter, you need to do the following:
Make sure there is a user who has permission to connect and perform common actions CRUD in the database.
Make sure the database is available for remote or local access.
To interact with the database it is necessary to:
1. Download the DB Connector Driver for specific DB (MySQL/MSSQL,etc)
2. Copy the DB Connector jar to ..\apache-jmeter-3.2\lib folder
3. Restart JMeter
4. Right click on Thread Group -> Add -> Config Element -> JDBC Connection Configuration
5. Fill in the Variable Name field. The value of this field is used to associate a specific connection configuration (JDBC Connection Configuration) to the database and a specific request (JDBC Request) sent by JMeter
6. Configure the JDBC Connection Configuration. Ex: jdbc:mysql://localhost:3306/DB_Name
7. JDBC driver class : com.mysql.jdbc.Driver
8. Provide User name and password for accessing the DB
JDBC Request:
1. Use the same variable name what given in JDBC Connection Configuration.
2. Set the Query Type. Ex: Select Statement
3. Result variable name – This variable will store all fields and field values received from the database.
If the variable name is blank.
java.lang.IllegalArgumentException: Variable Name must not be empty for element:JDBC Connection Configuration.
If the DB_Name is not given:
Response message: java.sql.SQLException: No database selected”
If the driver class was not found:
DataSourceElement: Could not load driver: {classname} java.lang.ClassNotFoundException: {classname}
If the database server is not running or is not accessible, then JMeter will report a java.net.ConnectException .
Clear Cache Each Iteration: The DNS Cache Manager caches the IP address -> the DNS hostname pairs and returns the cached value if the DNS hostname has a match in its internal cache. If this checkbox is ticked, the cache will be cleared at the start of the thread.
Use System DNS Resolver: The DNS Cache Manager will query DNS Server(s) defined in your OS network configuration in order to determine the hostname for each thread representing virtual users. In this case, you need to override the networkaddress.cache.ttl property in the $JAVA_HOME/jre/lib/security/java.security file and set it to 0 (the property defaults to “-1” which stands for “forever”)
Use Custom DNS Resolver: You can specify one or more DNS servers in the list.
If more than one server is provided, the DNS Cache Manager will choose a random one for each call – and will run as a round-robin mechanism.
If nothing is provided, the system DNS resolver will be used (the DNS server/s defined in your OS network configuration).
The DNS Cache Manager – Quick Tips and Troubleshooting
1. Where to put the DNS Cache Manager
The DNS Cache Manager should be added as a child of the Test Plan or a Thread Group element.
2. Always Use HTTPClient4
Make sure that you have HTTPClient4 selected in “Implementation” drop-down
3. Enable Logging
It’s easy to enable logging on the DNS Cache Manager. Just append the following parameter to the JMeter startup script:
-Ljmeter.protocol.http.control.DNSCacheManager=DEBUG
as
java -jar ApacheJMeter.jar -Ljmeter.protocol.http.control.DNSCacheManager=DEBUG
or
jmeter.bat -Ljmeter.protocol.http.control.DNSCacheManager=DEBUG
Keystore configuration element in JMeter helps to configure the Client’s side certificate. Some systems require a Client-side certificate which helps the server to know exactly who is connecting.
Before adding a Keystore Configuration element, you must set up a Java Key Store with the client certificates that you want to test. Follow the below steps:
#1 Create your certificates either with Java keytool utility or through your PKI and converting them to a format acceptable by JKS: If you have a PKCS12 file, use the following command line to convert it to a JKS file:
keytool -importkeystore -srckeystore certificate.p12 -srcstoretype PKCS12
-srcstorepass
-keystore -storepass
#2 Changes need to be done in system.properties file (if not available (or commented) then add (or uncomment) them:
javax.net.ssl.keyStore=path_to_keystore
javax.net.ssl.keyStorePassword=password_of_keystore
#3 Change your HTTP sampler implementation to Java (instead of HC3.1 or HC4)
Keystore Configurations:
Preload: This option helps you to choose whether you want preload Keystore or not?
Variable name holding certificate alias: Variable name that will contain the alias to use for authentication by client certificate.
Alias Start Index: The index of the first key to use in Keystore, 0-based
Alias End Index: The index of the last key to use in Keystore, 0-based. When using “Variable name holding certificate alias” ensure it is large enough so that all keys are loaded at startup.
The assertion in JMeter is used to validate the response of the request, that you have sent to the server. The assertion is a process where you verify the expected result with the actual result of the request at run time. If you need to apply assertion on a particular Sampler, then add it as a child of that Sampler.
Response Assertion – It facilitates the user by comparing the server response against a string pattern to check that the result is as expected.
Duration Assertion – Duration assertion is used to verify how long a request takes to complete. The Duration measured in milliseconds, and, if any request lasts longer than the value specified, the sample is marked as failed.
Size Assertion – It is to test that each response coming from server holds the expected number of bytes. It facilitates the user to specify the size.
XML Assertion – The XML Assertion checks that the returned response body is XML-compliant. Only the syntax is checked. Any external resources are neither downloaded nor validated.
HTML Assertion – The HTML Assertion checks that the response HTML syntax is a well-formed HTML/XHTML/XML document. So it is handy if your Web application or site requires zero HTML validation errors.
Beanshell Assertion:
String path = SampleResult.getURL().getPath();
if (!path.contains(“blazemeter”)) {
Failure = true;
FailureMessage = “URL Path: didn’t contain ‘blazemeter'” + System.getProperty(“line.separator”) + “URL Path detected: ” + path;
}
Compare Assertion:
The Compare Assertion checks the response content or confirms that the response time of all samplers under the assertion scope is equal.
User Parameter is a preprocessor whereas CSV Data Set Config and User Defined Variables are Config Elements.
In User Defined Variables, there is only one value of the variable can be defined whereas, in User Parameters and CSV Data Set Config, multiple values can be defined for a variable.
Test Data cannot be added via an external file in User Parameters and User Defined Variables whereas CSV Data Set Config use .csv file format as a test data.
Since there is only one value can be given in User Variables so bulk test data can not be used. The bulk test data can be added manually in User Parameters, but it is time-consuming. Although the primary purpose of CSV Data Set Config is to access an external file so that bulk test data can be available for the test.
The use of the regular expression is to extract the data from the responses in JMeter. It is also used in capturing dynamic value from the response.
The special characters above are −
( and ) − these enclose the portion of the match string to be returned
. − match any character
+ − one or more times
? − stop when first match succeeds
Template: $1$ to refers to group 1, $2$ to refers to group 2, etc. $0$ refers to whatever the entire expression matches.
Match No: 1 for first match, 2 for second match, 0 for random.
CSS Selectors are patterns to select elements with the following syntax:
.class
#id
* All elements
div, p all div and p elements
div p all p elements inside div elements
JQuery Selectors can just do the same as CSS Selectors, but with a different syntax:
$(“*”) All elements
$(“#id”)
$(“.class”)
$(“.class,.class”) — $(“.intro, .demo”) – all class elements with the class intro or demo
$(“h1,div,p”) all h1, div and p elements
option used for html response which for cleaning up malformed and faulty html. It doesn’t work with XML or XHTML response
Result Status Action Handler allows the user to stop the thread or whole test according to sampler result.
Here it defines the action to be taken after a sampler error.
Continue: This option ignores the error and continues the test.
Start next thread loop: It does not execute the test for current iteration when it found an error and restarts the loop on next iteration.
Stop Thread: In this option current thread exits.
Stop Test: In this option, the whole test is stopped at the end of any current sample.
Stop Test Now: The entire test is stopped here. Any current samplers are interrupted if possible.
Java applications obtain objects in memory as needed. It is the task of garbage collection (GC) in the Java virtual machine (JVM) to automatically determine what memory is no longer being used by a Java application and to recycle this memory for other uses.
Java garbage collection is the process by which Java programs perform automatic memory management. The garbage collector finds these unused objects and deletes them to free up memory.
What are the different types of garbage collectors in Java?
- Serial Garbage Collector – it works for standalone application. If you use serial GC for distributed applications, u will observe high performance issues.
- Parallel Garbage Collector.
- CMS Garbage Collector (Concurrent Mark Sweep).
- G1 Garbage Collector.
The Java heap is the area of memory used to store objects instantiated by applications running on the JVM. Objects in the heap can be shared between threads. Many users restrict the Java heap size to 2-8 GB in order to minimize garbage collection pauses.
Heap memory should not be greater than physical memory on your system.
Min size of heap memory can be 1/64th sized of system physical memory.
Max size of heap memory can be 1/4th size of system physical memory.
Jave Heap contains New/Young Generation and Old/Tenured Generation.
In New/Young Generation, we have Eden and Survival space. Survivor has two survivor spaces S0 and S1.
Till JDK7, we have perm gen space. From JDK8 onwards, this has been converted to metaspace.
Increasing the heap memory doesn’t fix the outofmemory exception. It might reproduce after sometime.
Increasing the heap size, will improve the response time.
When heap size is increased, GC activities will increase (Minor/Major/Full GC).
When GC activities increases, it suspends all threads for the time GC is running and it impacts the total response time.
| To check the default Heap and PermGen sizes | Run this command: java -XX:+PrintFlagsFinal -version | findstr /i “HeapSize PermSize ThreadStackSize” |
| How to check which GC is running on your system? | Run this command: java -XX:+PrintCommandLineFlags -version |
| To change from Parallel GC mode to G1GC mode | Run this command: java -XX:+UseG1GC -XX:+PrintCommandLineFlags – version |
| Garbage collection happens in three phases | 1. Marking – Marks all the objects which are referenced/reachable 2. Delete / Sweep – deletes all the unreachable objects 3. Compaction – brings the used memory to one side and free to other side |
| Why should we use G1GC | It will go to either Young, Old or Perm Gen space, whichever is full and performs GC activity. |
| Serial GC | It is available right from the first version of JAVA. It is single threaded, Mark and Sweep Collector. USed for small applications. Disadvantages: It pauses all application threads whenever it is working. It is single threaded. When should we use Serial GC: 1. It is used for small amount of heap (approx. 100MB) 2. It runs in single thread. It is best suited to single processor machine 3. Application with medium sized to large sized data sets that run on multiprocessor or multithreaded hardware. |
| Parallel Collector | It is similar to Serial Collector but multiple threads are used to speedup garbage collection. Does not kick in until heap is full or almost full. Stops the world pause when runs. When should we use Parallel GC: If peak application performance (throughput) is the priority. If there are no pause time requirements or pauses of 1 second or longer are acceptable. |
| Concurrent Mark Sweep (CMS) Collector | Iniitial Mark – pauses all application threads. It will identify the live objects which are connected to GC roots. Concurrent Mark – It identifies objects which are directly or indirectly connected to the initially markd live objects. Concurrent Preclean – If any new objects created during the above process. Re-mark – Pauses all application threads. Usually longer than the initial mark. Sweep – all unreach objects are sweeped. Concurent Reset – resets the heap memory Works on old generation. It works when the old generation is amost 80% full. Advantages: very low pause time; as many phases run concurrently with application. Disadvantages: It causes heap fragmentation, as there is no compacting phase. Concurrent mode failure: If the CMS collector is unable to finish reclaiming the unreachable objects before the tenured generation fills up. If the CMS is unable to move the objects from young generation to tenured generation and there is no space for this object in tenured generation, this is called concurrent mode failure. When should we use Serial GC: If application response time is more important than overall throughput. If garbage collector pauses must be kept shorter than 1 sec. |
| G1 GC | New in JAVA 6. Officially supported in Java 7. G1 is planned as long term replacement for CMS. G1 supports heaps larger than 4 GB. and is parallel, concurrent and incrementally compacting low pause garbage collector. With G1, heap is partitioned into a set of equal size heap regions. A total of approx. 2000 regions, in which each region would be of size 1MB to 32 MB. determined by JVM. The principle of G1GC is to reclaim the JAVA heap by collecting the region with mostly occupied unreachable objects, hence the name Garbage first. Evacuation – Live objects are moved from young generation either into survivor regions or old/tenured generation. Remembered Sets or RSets – track object references into a given region. There is one RSet per region in the heap. Collection Sets or Csets the set of regions that will be collected in a GC. CSet is evacuated (copied/moved) during a GC. Set of regions can be Eden, Survivor, and/or old generation. When should we use Serial GC: 1. Large heapsize. 2. Full GC durations are too long or too frequent 3. The rate of object allocation rate or promotion varies significantly 4. Undesired long garbage collection or compaction pauses (longer than 0.5 to 1 sec) |
Preconditions:
Make sure that you will have the same JMeter version (always use the latest one) on all machines (slaves and master).
Confirm that master and slaves are under the same subnet.
Assure that firewalls on the operative systems are turned off.
It is recommended to have the same java version on all machines (master and slaves).
Check that all slaves have the plugins that will be used by the test script that will have its execution triggered by the master node. Master sends the .jmx file to its slaves but it does not send the required plugins.
Test data files (such as .csv files) also need to be manually copied to the slaves once it is also not done by the master.
Slaves Setup
RMI Keystore Generation
Since JMeter 4.0, the default transport mechanism for RMI will use SSL by default. If you don’t want to use it, then it will be needed to open the jmeter.properties file, uncomment the line where server.rmi.ssl.disable property is located and change it from false to true.
server.rmi.ssl.disable=false
Configuring the slaves’ firewalls
By default, RMI uses dynamic ports for the JMeter server engine. This can cause problems for firewalls once you would need to allow incoming connections from different ports every time you start the jmeter-server script (.bat for windows and .sh for Unix) on a JMeter slave.
In order to avoid using random ports, you will need to set a specific value:
server.rmi.localport=4000
Master Setup
Configuring the master node is simpler than doing that for the slave nodes. You just need to open the jmeter.properties file, find the remote_hosts property, uncomment it and type the ip addresses
remote_hosts=172.10.0.40
Starting Slaves and Master
We are almost there, but before executing the test script we just need to initialize the slaves and master. In order to start a slave (or server) you need to go to the JMeter/bin folder, find the jmeter-server (.bat on Windows and .sh on Unix) file and execute it.
Starting the master is pretty straight forward. Just go to jmeter/bin and open jmeter (.jar or .bat on Windows and .sh on Unix). Open a .jmx file and go to Run >> Remote Start All
Limitations
RMI cannot communicate across subnets without a proxy; therefore neither can jmeter without a proxy.
Since JMeter sends all the test results to the controlling console, it is easy to saturate the network IO. It is a good idea to use the simple data writer to save the results and view the file later with one of the graph listeners.
Unless the server is a large multiprocessor system, in most cases 1–2 clients is sufficient to overwhelm the server.
Processor is used to modify the Samplers in their scope.
There are 2 Types of processors:
Pre-processor – Pre-processor executes some action before making Sampler Request.
Post-processor – Pos-Processor will be executed when a Sampler Request finishes its execution.
Post-Processor in JMeter
1. Regular Expression Extractor: – Regular Expression Extractor is used to extract values from the response of test server using a Perl-Type Regular Expression.
2. XPath Extractor: – XPath Extractor element is used to extract values from structured response XML or (X)HTML by using XPath query language. use Tidy to parse HTML response into XHTML.
3. Result Status Action Handler: – Result Status Action Handler element can be used to stop the thread or the test if the relevant sampler gets failed.
Pre-Processor in JMeter
A Post-Processor executes after a sampler finishes its execution. This element is most often used to process the response data.
The Transaction Controller generates an additional sample which measures the overall time taken to perform the nested test elements.
When the check box “Include duration of timer and pre-post processors in generated sample” is checked, the time includes all processing within the controller scope, not just the samples.
The Throughput Controller allows the user to control how often it is executed. There are two modes – percent execution and total executions.
Percent executions causes the controller to execute a certain percentage of the iterations through the test plan. Total executions causes the controller to stop executing after a certain number of executions have occurred.
In JMeter, spike testing can be achieved by using Synchronizing Timer. The threads are jammed by synchronizing the timer until a specific number of threads have been successfully blocked, and then release them at once thus creating large immediate load.
Correlation is the process of capturing and storing the dynamic response from the server and passing it on to subsequent requests. … Correlation is a critical process during performance load test scripting, because if it isn’t handled correctly, our script will become useless.
- Configuration elements
- Pre-Processors
- Timers
- Sampler
- Post-Processors (unless SampleResult is null)
- Assertions (unless SampleResult is null)
- Listeners (unless SampleResult is null)
Pacing in load testing refers to the time between the iterations of your test scenarios. This is unlike Think Time, which refers to the delay between actions or interactions inside iterations. Pacing allows the load test to better simulate the time gap between two sessions.
By using pacing in your test, you will be able to regulate the rate of requests that hit your application and accurately achieve a desired load. This will allow you find out the exact load capacity your application can handle.
Below timers are used to implement pacing in Jmeter to achieve the desired load/Business transactions.
1.Precise Throughput Timer
2.Constant Throughput Timer
3.Throughput Shaping Timer
Think Time: The wait time between transactions is called think time.
Pacing Time: The wait time between iterations is called Pacing Time.
The first thing which you need to check is ‘Type of Error‘. Without knowing the type you can not investigate. During the performance test, you may get different types of error like client-side error, server-side error etc.
Scriptings Errors:
a). Missed to replace old URLs in some places: This causes the failure of a particular transaction.
b). Lack of test data: Sometimes tester forgets to set “recycle the test data list” or “continue with last value”
c). LG failed during the test: This issue occurs when LG(s) are unable to handle user load during the test.
d). LG memory issue: This issue comes when there is no enough memory in LG. LG may get disconnected in between the test.
e). HTTP 4XX: Refer response code graph, if the test result having HTTP 4XX error then this indicates something wrong has been sent to the server.
HTTP 4XX followed by 5XX: Again this is a syntax of client-side error, but not always. This is a tricky scenario where you have to carefully analyse the previous requests having 4XX response code and the following request having 5XX response code.
Following are the tricks that help in reducing resource usage.
1. Use a non-GUI mode.
jmeter -n -t test.jmx -l test.jtl
2. It is better to use as few Listeners as possible. Applying the “-l” flag as shown in above point may delete or disable all the Listeners.
3. Disable the “View Result Tree” listener as it consumes a lot of memory and may result in JMeter tool running out of memory. It will freeze the console too. It is, however, safe to use the “View Result Tree” listener with only “Errors” kept checked.
4. Instead of using a similar Sampler a large number of times, use the same Sampler in a loop and use variables (CSV Data Set) to vary the sample data. Or perhaps use the Access Log Sampler.
5. Avoid using functional mode.
6. Use CSV output rather than XML.
Also, you may like to read some of the common points.
7. Try to save the data that you need.
8. Use as few Assertions as possible.
9. Disable all JMeter graphs as they consume a lot of memory. All the real-time graphs can be viewed using the JTL tab in the web interface.
10. Do not forget to erase the local path from CSV Data Set Config when used.
11. Cleaning of the Files tab before every test run.
JMeter has all the capabilities to behave like a real browser and still maintain high scalability. For this, the following config elements should be added to JMeter:
A cache manager to simulate the browser’s cache.
A cookie manager to simulate the browser’s cookies.
In the header manager include a user agent line. (We recommend recording from your own browser).
Use thread/connection pool to simulate the browser parallel fetching (use between 2-4).
Ask JMeter to retrieve embedded resources to simulate a browser retrieving embedded resources (such as gifs, css, js etc).
In Apache JMeter, we can simulate network bandwidth very easily.
Following are the steps to simulate network bandwidth:
Open ‘jmeter.properties’ file in notepad (location: apache-jmeter/bin)
Search ‘cps’ (characters per second) you will find
Define characters per second > 0 to emulate slow connections
#httpclient.socket.http.cps=0
#httpclient.socket.https.cps=0
Change cps value as per your need
cps = rb*128 (rb= Required Bandwidth)
cps = (target bandwidth in kbps * 1024) / 8
| Bandwidth | cps Value |
| GPRS | 21888 |
| 3G | 2688000 |
| 4G | 19200000 |
| WIFI 802.11a/g | 6912000 |
| ADSL | 1024000 |
| 100 Mb LAN | 12800000 |
| Gigabit Lan | 128000000 |