Benchmarks
Let's talk numbers!
The tests
To get a glance of Appwrite performance when deploying it I'll run a simple load-tests using oha.
In all the tests I'll check the RPS - requests per seconds — for the following actions:
- Adding a document to database
- Reading a document from the database
- Fetching a bunch of documents
- Executing two types of functions
- The first one will just return JSON with no other logic involved.
- The second one will fetch a bunch of documents from the database.
In each comparison, I'll also compare the PPR - Price Per Request — based on the selected cloud pricing. This is not a rocket-science, but another way to easily understand the server cost compare to requests.
Vertical Scaling
Between each test, the server has been restored to a snapshot. The tests were done using Hetzner servers.
Each server specs shows the number of shared-vCPU cores and the GB of RAM.
| Server | Adding | Reading | Fetching | E1 | E2 | Price | PPR |
|---|---|---|---|---|---|---|---|
| 2/2 | 170 | 427 | 241 | 106 | 80 | 3.85€ | 0.00375 |
| 4/8 | 443 | 870 | 511 | 288 | 180 | 13.10€ | 0.00571 |
| 8/16 | 939 | 1,857 | 1,015 | 436 | 235 | 24.70€ | 0.00551 |
| 16/32 | 2,140 | 3,841 | 2,106 | 1,311 | 519 | 54.40€ | 0.00549 |
Even in a 2CPU/2GB RAM Appwrite is able to insert 170 documents per second. That's not just a direct insert to the database, but a whole request that goes through all Appwrite init, permissions and other modules. It can be translated to 14,382,000 requests in 24 hours, that's impressive!
While in the strongest 16CPU+32GB RAM you got 2,140 per second, which can be translated to 181,044,000 insertions per day! Wow.
Vertical Scaling Decentralizes
In this deployment, Appwrite was deployed on a 2-server infrastructure
- 1CPU/1GB 6$ server holding the databases and storage.
- Vertically scaled server from 2/2 up to 16/32
The only appeared disadvantage is when reading single data from the Database. But, the options this method gives toward horizontal scaling and having zero-downtime make it really attract to start with.
Horizontal Scaling Decentralizes
In this deployment, Appwrite was deployed to at least 3 servers using Hetzner cloud.
- Database & Storage - 8CPU/16GB 24€ server
- Load balancer - 5.39€ server
- Services - 4CPU/8GB 13.10€ server
The First number on the table represents the number of instances of the Services server.
| # | Adding | Reading | Fetching | E1 | E2 | Price | PPR |
|---|---|---|---|---|---|---|---|
| 3 | 833 | 2529 | 1668 | 630 | 398 | 68.69€ | 0.00957 |
| 5 | 1031 | 4065 | 2284 | 1065 | 630 | 94.89€ | 0.00815 |
| 9 | 826 | 5375 | 2810 | 1341 | 784 | 147.29€ | 0.00951 |
| 15 | 721 | 6928 | 4340 | 1577 | 1102 | 225.89€ | 0.01049 |
I'm aware of the Adding mismatch for the 15-nodes or in 5-nodes HA deployment, as of now it's a bit of a mystery.
Docker Swarm
In this deployment, Appwrite was deployed to at least 6 servers using Hetzner cloud.
- Database & Storage - 8CPU/16GB 24€ server
- Load balancer - 5.39€ server
- ⚡ Swarm Manager - 4CPU/8GB 13.10€ server
- ☁️ Swarm worker - 4CPU/8GB 13.10€ server
| ⚡ | ☁️ | Adding | Reading | Fetching | E1 | E2 | Price | PPR |
|---|---|---|---|---|---|---|---|---|
| 1 | 3 | 1631 | 3270 | 2030 | 953 | 126 | 81.79€ | 0.00771 |
| 3 | 5 | 2217 | 5042 | 2745 | 1211 | 150 | 134.19€ | 0.00862 |
| 3 | 9 | 3464 | 6552 | 2900 | 1194 | 610 | 186.59€ | 0.00883 |
Why the functions measure so low?
When you're executing a function Appwrite is sending a request to the executor container, the executor is checking to see if the OpenRuntime container is already deployed, and if not it will be created.
In order for the executor to be able to create, deploy and delete containers, is being bind with the current node docker.socket. That Docker context is not in the Swarm level but at the local node level.
For that reason you can have only one executor and appwrite-worker-functions and they both need to be on the same node.
You can try to mitigate it by putting those containers on a powerful server specific for them.
What deployment type to choose?
For most use-case, it will probably be best to start with Vertical + Decentralized infrastructure, this way, when upgrading you won't get 100% downtime.
If you sure this is a small project, you can go single server.
Docker Swarm and Horizontal + Decentralized are mostly the same in terms of performance, but remember that:
- Horizontal has an advantage when aiming for High availability as all the services are close to the end user.
- Swarm has an advantage as it is much easier to scale up and down the services.