47 lines
1.6 KiB
Markdown
47 lines
1.6 KiB
Markdown
+++
|
|
title = "Ceph Benchmarking"
|
|
date = 2026-03-01
|
|
description = "The results of some of my recent ceph benchmarks"
|
|
draft = true
|
|
|
|
[taxonomies]
|
|
categories = ["Homelab"]
|
|
tags = ["Homelab", "Ceph"]
|
|
+++
|
|
|
|
## Setup everything for the Benchmarking
|
|
On a machine in the ceph-cluster already:
|
|
1. Generate a minimal ceph config using `ceph config generate-minimal-conf`
|
|
2. Create a user for Benchmarking
|
|
1. Create a new user `ceph auth add client.linux_pc`
|
|
2. Edit the caps for my use-case
|
|
- `mgr`: `profile rbd pool=test-rbd`
|
|
- `mon`: `profile rbd`
|
|
- `osd`: `profile rbd pool=test-rbd`
|
|
3. Get the keyring configuration using `ceph auth get client.linux_pc`
|
|
|
|
On the client machine doing the Benchmarking:
|
|
1. Install basic ceph tools `apt-get install -y ceph-common`
|
|
2. Load the rbd kernel module `modprobe rbd`
|
|
3. Setup local ceph config
|
|
- Copy the generated configuration to `/etc/ceph/ceph.conf`
|
|
- `chmod 644 /etc/ceph/ceph.conf`
|
|
4. Setup local ceph keyring
|
|
- Copy the keyring configuratio nto `/etc/ceph/ceph.client.linux_pc.keyring`
|
|
- `chmod 644 /etc/ceph/ceph.client.linux_pc.keyring`
|
|
5. Confirm your configuration is working by running `ceph -s -n client.linux_pc`
|
|
|
|
Setup the benchmark itself:
|
|
1. `rbd create -n client.linux_pc --size 10G --pool test-rbd bench-volume`
|
|
2. `rbd -n client.linux_pc device map --pool test-rbd bench-volume` (which should create a new block device, likely `/dev/rbd0`)
|
|
3. `mkfs.ext4 /dev/rbd0`
|
|
4. `mkdir /mnt/bench`
|
|
5. `mount /dev/rbd0 /mnt/bench`
|
|
|
|
## Benchmarks
|
|
|
|
## TODO
|
|
- Try directly on the block device
|
|
- Try this using xfs instead of ext4
|
|
- Try this with and without drive caches
|