Testing gRPC #5: Load test a gRPC service
Learn how to write a load test for a locally hosted gRPC Java service using locust4j and run it on a local locust python-based master service
This is the final blog in this series where we explored gRPC testing and automation.
A quick recap, In the previous blogs in this series, we explored testing different layers of the test pyramid within the gRPC context:
How to unit test the server
How to unit test the client code.
How to E2E test a server method
Please feel free to catch up on those before reading this one.
What you’ll learn? 🌱
In this blog, I’ll explore how to set up a load test written in Java leveraging the locust4j library to set up a locust slave.
Our slave worker will push aggregated results to a Python-based locust server.
I hope you are also as excited as I am to explore this topic.
Let’s go 🏃
Why write load tests?
Any modern successful business is bound to hit the scale at some point in its journey. When it does, you don’t want to be caught with a bad surprise wherein your systems are not able to scale or handle faults gracefully.
Load testing prepares engineering teams for such events.
Identifying how the system holds up with the next level of scale is extremely important.
A system that is not scalable, reliable, or fault-tolerant when put under stress would let down its users and often that means direct revenue loss or even worse losing a customer’s trust.
Preventing such bugs from leaking to production should be an acceptance criterion for any service seeing the light of day in production.
⚓ Setup locust master
I hope we are on the same page that writing load tests is super important.
Let’s focus on understanding how to do this for our route guide service
We will use a popular load testing framework called locust for this example. I had previously explored how to write a locust Python test on a mock API. Please feel free to give it a read if interested
How to set a load test via Locust - automation hacks
We will need python3 to be installed on our machine.
Most modern mac comes with python3 pre-installed, if not, you can follow this wonderful guide to Properly Installing Python
We will also set up virtualenv to install the required dependency, in this case, locust, and will use Pipenv. You can read How to manage your Python virtualenvs with Pipenv - Automation Hacks to get familiar with Pipenv
Below is a summary:
Ensure you have the below setup in your .zshrc, .bash_profile, or in your Windows system environment variables
# One time setup
# Set below in .zshrc or .bash_profile depending upon the type of shell you are using
export WORKON_HOME=~/virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8Next, install pipenv using python3
pip3 install pipenvYou can then clone or take a pull from this repository GitHub - automationhacks/grasp-grpc: Ports routeguide example service from grpc-java repository and adds functional and non-functional API tests and ensure you cd to folder with Pipfile
Run below to start virtualenv
pipenv shellFinally, we’ll start our locust master server using:
locust -f locust_master.py --master --master-bind-host=127.0.0.1 --master-bind-port=5557You’ll see something like this
[2024-05-31 08:29:15,073] Gauravs-Laptop/INFO/locust.main: Starting web interface at http://0.0.0.0:8089
[2024-05-31 08:29:15,078] Gauravs-Laptop/INFO/locust.main: Starting Locust 2.28.0What does this command do?
Here we are running a dummy locust server in master mode and binding the server to localhost and a random port 5557. In a production setup, this script could be running on a remote server which we would use in our tests.
Below is what locust_master.py looks like:
from locust import User, TaskSet, task
class DummyTask(TaskSet):
@task(1)
def dummy(self):
pass
class Dummy(User):
tasks = [DummyTask]Here we init DummyTask as a subclass of TaskSet and have a method dummy with a weight of 1 using the @task() annotation.
Weight is a way in locust to increase the probability of a certain task being picked ahead of others. In this case, we have a single task so it does not matter.
class DummyTask(TaskSet):
@task(1)
def dummy(self):
passWe also have the Dummy class of type User with our tasks list with this DummyTask
class Dummy(User):
tasks = [DummyTask]Generate required files using proto proto-compiler
Ensure all relevant files are generated using the gRPC proto compiler by executing below.
./gradlew installDistYou only need to run this if you've made some changes to the proto message or RPC. This step could be skipped if there are no changes. In a prod setup, you may have an automation process that updates dependencies as part of a CI/CD pipeline
Start gRPC server
Next, let’s ensure our server and API are up by executing the below
./build/install/grasp-grpc/bin/route-guide-serverYou’ll see something like this
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
May 31, 2024 8:34:47 AM io.automationhacks.routeguide.RouteGuideServer start
INFO: Server started, listening on 8980Writing a locust slave
The key component of this load test is a locust slave written in Java that can be used by locust4j to start the load test.
I’ve added an example slave in RouteGuideLoadGen.java
It looks like below:
package io.automationhacks.routeguide.perf;
import com.github.myzhan.locust4j.Locust;
import io.automationhacks.routeguide.perf.tasks.GetFeatureTask;
import static io.automationhacks.routeguide.constants.Constants.*;
public class RouteGuideLoadGen {
public void getFeatureLoad() {
var locust = configureLocustMaster();
locust.run(new GetFeatureTask(1));
}
private Locust configureLocustMaster() {
var locust = Locust.getInstance();
locust.setMasterHost(LOCUST_MASTER_HOST);
locust.setMasterPort(LOCUST_MASTER_PORT);
locust.setMaxRPS(LOCUST_MAX_RPS);
return locust;
}
public static void main(String[] args) {
new RouteGuideLoadGen().getFeatureLoad();
}
}Let’s unpack this
We have a method to configure our slave worker with the locust master instance
We then invoke the run method and provide it a task instance
public void getFeatureLoad() {
var locust = configureLocustMaster();
locust.run(new GetFeatureTask(1));
}We also use the below method to get a Locust instance and set the master host, port as well as peak requests per second (RPS) for the slave.
private Locust configureLocustMaster() {
var locust = Locust.getInstance();
locust.setMasterHost(LOCUST_MASTER_HOST);
locust.setMasterPort(LOCUST_MASTER_PORT);
locust.setMaxRPS(LOCUST_MAX_RPS);
return locust;
}Locust load task
We also have to write a Locust task that has the actual gRPC API that we want to invoke.
public class GetFeatureTask extends AbstractTask {
private final int weight;
RouteGuideTestClient client =
new RouteGuideTestClient(ROUTE_GUIDE_SERVER_HOST, ROUTE_GUIDE_SERVER_PORT);
public GetFeatureTask(int weight) {
this.weight = weight;
}
@Override
public int getWeight() {
return weight;
}
public String getName() {
return "RouteGuide.GetFeature";
}
public void execute() {
int latitude = 407838351;
int longitude = -746143763;
Point point = Point.newBuilder().setLatitude(latitude).setLongitude(longitude).build();
// Record the startTime for the API call
var startTime = getTime();
// Add thread safe logic here to make the API call
Feature response = client.getFeature(point);
// Record the endTime for the API call
var endTime = getTime();
// Do not swallow any exceptions from the API, we should propagate any exceptions to locust
assertWithMessage(
"Could not find the feature at lat: %s long: %s".formatted(latitude, longitude))
.that(response.getName())
.isEqualTo("Patriots Path, Mendham, NJ 07945, USA");
// Record success or failure to locust
long responseTime = endTime - startTime;
int contentLength = response.getName().length();
Locust.getInstance()
.recordSuccess("GetFeature", response.getName(), responseTime, contentLength);
}
}Above is the complete method, let’s understand how this works
We create a task GetFeatureTask that extends AbstractTask from locust4j
We configure the local Grpc client
RouteGuideTestClient client =
new RouteGuideTestClient(ROUTE_GUIDE_SERVER_HOST, ROUTE_GUIDE_SERVER_PORT);And also configure a constructor and getter to set custom weight if we want from the caller
public GetFeatureTask(int weight) {
this.weight = weight;
}
@Override
public int getWeight() {
return weight;
}We set the name to be used during the load test
public String getName() {
return "RouteGuide.GetFeature";
}Finally, the execute method has the actual logic wherein we invoke the locally running service
public void execute() {
int latitude = 407838351;
int longitude = -746143763;
Point point = Point.newBuilder().setLatitude(latitude).setLongitude(longitude).build();
// Record the startTime for the API call
var startTime = getTime();
// Add thread safe logic here to make the API call
Feature response = client.getFeature(point);
// Record the endTime for the API call
var endTime = getTime();
// Do not swallow any exceptions from the API, we should propagate any exceptions to locust
assertWithMessage(
"Could not find the feature at lat: %s long: %s".formatted(latitude, longitude))
.that(response.getName())
.isEqualTo("Patriots Path, Mendham, NJ 07945, USA");
// Record success or failure to locust
long responseTime = endTime - startTime;
int contentLength = response.getName().length();
Locust.getInstance()
.recordSuccess("GetFeature", response.getName(), responseTime, contentLength);
}
An interesting thing to note is that we capture the total time taken for the API call and response to come back and then later on persist it in the Locust instance using below:
Locust.getInstance()
.recordSuccess("GetFeature", response.getName(), responseTime, contentLength);Start the load test
We can run RouteGuideLoadGen.java to start the slave and register it with the master.
If we navigate to the locust master host running at
http://0.0.0.0:8089, we can see below the UI
Here we’ll be kind and just use 5 users with a ramp up of 1 user per second.
We can see the charts with real-time performance RPS failure stats and other diagnostic data.
The worker’s tab also shows the current no of workers registered
And we can also see stats about the overall test
Since this is a local test running on my machine, I also used the Activity monitor app and could see a spike in CPU and memory usage stats while the test was running. Within a short time, I bombarded 40K RPS on my locally hosted service. Surely a pretty sizable scale for a single service. 🤩
References
Standing on the shoulders of giants.
I referred to the below resources to come up with this write-up. Hopefully, this is useful for your further exploration.
Distributed Load Testing of REST/gRPC APIs using Locust
Locust Http load example project
Locust Http load is another example project
Conclusion
This was a short introduction to the world of load testing with locust and demos and how you are not limited to knowing Python to leverage this wonderful tool. I hope you’ll give it a try and explore the nitty-gritties of this tool
To summarize, we learned:
We can write a locust load test for a gRPC service using Java with a slave to Locust master setup
How to set locust master in Python
How to generate the required build file for the Java gRPC service
Start a local gRPC server
Write a utility to set a Locust Java slave worker
Start execution and observe metrics on Locust web UI
Please let me know if you have questions or thoughts in the comments.
Thanks for the time you spent reading this 🙌. If you found this post helpful, please subscribe to the substack newsletter and follow my YouTube channel (@automationhacks) for more such insights in Software Testing and Automation. Until next time 👋, Happy Testing 🕵🏻 and Learning! 🌱| Newsletter | YouTube | Blog | LinkedIn | Twitter.