-
Notifications
You must be signed in to change notification settings - Fork 475
OBP Performance Tests
This document aims to answer the question: “How do we serve X thousand concurrent Users using OBP?”
Thus we first need to define: “What do we mean by a concurrent user?”
Let's define a User as: “A user using a mobile App for 5 minutes and making 10 API calls.”
An average user session lasts 5 minutes.
|-----|
In an hour we will have 60 / 5 = 12 of these 5 min sessions:
|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|
So our famous “Concurrent User” will make 120 calls in an hour.
So if we want to support 50,000 concurrent users: We will handle 50,000 * 120 = 6,000,000 requests per hour
i.e. 50,000 concurrent users... Make 1666 requests per second.
We want to support an average response time of 500ms
So the question is: “How many OBP instances do we need to support this?”
To measure performance of an OBP API system, we use a JMeter test plan that simulates a typical client facing Application making a range of HTTP requests (calls) to the API.
This Test Plan reflects a simple AISP (Transaction history) and PISP (Payment) scenario.
Although authentication is not typically part of the main OBP-API responsibility we include a simple authentication call for ease of running the tests.
DirectLogin: Login with username and password, generate a login token. GetCurrentUser: Gets details about the currently logged in user GetAccountsAtBank: List all bank accounts the user has GetBalances: Gets balances of all bank accounts of the user GetTransactions: Gets transaction history of one bank account CreateTransactionRequest: Initiates a bank transfer
In between these API calls, there are random pauses; the runtime of the entire simulated app was set to between 30s and 75s.
The OBP instance tested consisted of a single-host installation with all components on one virtual machine: OBP API, database backend (PostgreSQL), and reverse http proxy (NGINX). The VM had 4 vCPU and 8GB of RAM.
The test plan was configured to run over roughly 4 minutes during which a total number of 500 simultaneous iterations of the simulated app were run from a separate JMeter VM over a 1000MBit/s network connection. Each iteration was using a different user account on the instance. This test plan was executed several times, with similar results every time.
The response times during one execution averaged at about 100ms per API call, detailed metrics are as follows:
A near linear scaling of this setup can be assumed, as all the components can be run on separate and dedicated VMs or containers. I.e. If the load balancers, database cluster and OBP API instances all run with dedicated resources, they will be able to perform their dedicated tasks.