Scribd
Upload
7 views8 pages

Zfs 3

The document outlines a benchmarking procedure for ZFS synchronous read performance, focusing on optimizing settings for improved throughput and reduced CPU usage. It details commands for configuring ZFS parameters, running tests with FIO, and analyzing results, which showed an increase in read speed from 218 MiB/s to 252 MiB/s after applying tunable optimizations. Key optimizations include adjusting record size, enabling metadata-only caching, and configuring ARC size to enhance performance for high-concurrency workloads.

Uploaded by

yomaxi5211
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views8 pages

Zfs 3

The document outlines a benchmarking procedure for ZFS synchronous read performance, focusing on optimizing settings for improved throughput and reduced CPU usage. It details commands for configuring ZFS parameters, running tests with FIO, and analyzing results, which showed an increase in read speed from 218 MiB/s to 252 MiB/s after applying tunable optimizations. Key optimizations include adjusting record size, enabling metadata-only caching, and configuring ARC size to enhance performance for high-concurrency workloads.

Uploaded by

yomaxi5211
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Read/Write ZFS Performance

Synchronous Reads

Purpose
This command benchmarks sequential read performance for a high-concurrency workload
(2 jobs, 32 I/O depth) with a controlled throughput rate (125 MiB/s). It simulates scenarios
like large file reads (e.g., video streaming) in an asynchronous I/O environment.

Simple Comparison Workflow

1. Run Benchmark with Default Settings


No changes to ZFS tunables—benchmark the default configuration.

Command

1. Set Record Size to 128 KiB


sudo zfs set recordsize=128k Docker

2. Enable Metadata-Only Caching


sudo zfs set primarycache=metadata Docker

3. Disable Prefetching
echo 1 | sudo tee /sys/module/zfs/parameters/zfs_prefetch_disable > /dev/null

4. Default ARC Size


echo 0 | sudo tee /sys/module/zfs/parameters/zfs_arc_max > /dev/null

5. Run FIO
sudo fio --name=test_seq_read_async --rw=read --bs=1M --size=1G --numjobs=2
--runtime=120 --time_based --rate=125m --iodepth=32 --ioengine=libaio
--directory=/var/lib/docker --fallocate=none --group_reporting

Command Explanation
● --rw=read: Runs a read operation.
● --ioengine=sync: Forces synchronous I/O.
● --bs=4k: Block size is 4 KiB.
● --size=1G: Test 1 GiB of data.
Results
Look at the outputs from the fio tests for the following metrics.
1. Throughput (READ in MiB/s): Check the READ line for throughput.
2. IOPS: Check the iops value.
3. CPU Usage: Look at cpu usage.
4. Total Data Read: Look for total read data.

Non-Tunable Table

Grafana Results

Throughput
node_zfs_zpool_dataset_reads

Operation: Rate

The read speed comes out to be 218 MiB/s


CPU Usage
node_cpu_seconds_total

Operation1: Rate Operation2: Avg


Label Filters: Mode, System

The CPU usage is 34.00%

node_cpu_seconds_total

Operation1: Rate Operation2: Avg


Label Filters: Mode, User

The CPU usage is 2.04%


Total Data Read
node_disk_read_bytes_total
Operation1: Increase (Range 1h) Operation2: Divide By Scalar (Value 1000000000)

The total data read is 26.4 GB

2. Apply Tunable Optimizations


Tunable Setting are applied

Command

1. Set Record Size to 128 KiB


sudo zfs set recordsize=128k Docker

2. Enable All-Only Caching


sudo zfs set primarycache=all Docker
sudo zfs set secondarycache=all Docker

3. Enable Prefetching
echo 0 | sudo tee /sys/module/zfs/parameters/zfs_prefetch_disable > /dev/null

4. Adjust ARC Size (50%-60% Of RAM)


echo 3221225472 | sudo tee /sys/module/zfs/parameters/zfs_arc_max > /dev/null

5. Set Logbias To Throughput


sudo zfs set logbias=throughput Docker

6. Disable Sync For Non-Critical Writes


sudo zfs set sync=disabled Docker

7. Remove previous files


sudo rm /var/lib/docker/test_seq_read_async.0.0
/var/lib/docker/test_seq_read_async.1.0
8. Run FIO
sudo fio --name=test_seq_read_async --rw=read --bs=1M --size=1G --numjobs=2
--runtime=120 --time_based --rate=125m --iodepth=32 --ioengine=libaio
--directory=/var/lib/docker --fallocate=none --group_reporting

Compare Results
Look at the outputs from the fio tests for the following metrics.
1. Throughput (READ in MiB/s): Check the READ line for throughput.
2. IOPS: Check the iops value.
3. CPU Usage: Look at cpu usage.
4. Total Data Read: Look for total read data.

Comparison Table

Grafana Results

Throughput
node_zfs_zpool_dataset_reads

Operation: Rate

The read speed comes out to be 252 MiB/s


CPU Usage
node_cpu_seconds_total

Operation1: Rate Operation2: Avg


Label Filters: Mode, System

The CPU usage is 26.8%

node_cpu_seconds_total

Operation1: Rate Operation2: Avg


Label Filters: Mode, User

The CPU usage is 1.6%


Total Data Read
node_disk_read_bytes_total
Operation1: Increase (Range 10m) Operation2: Divide By Scalar (Value 1000000000)

The total data read is 31.2 GB

3. Explanation

Recordsize Adjustment
The recordsize parameter determines the block size used by
ZFS for reading and writing data. By default, it is set to 128 KiB, which is optimized for
larger, sequential workloads like backups or video streaming. For synchronous reads,
which often involve small, random I/O operations (e.g., database queries), a smaller
recordsize (e.g., 4 KiB) is more suitable. This matches the typical block size of such
workloads and reduces read amplification, meaning ZFS doesn’t fetch more data than
necessary. The result is improved IOPS and reduced latency, as the system processes
only the required data.

Metadata-Only Caching
ZFS uses the Adaptive Replacement Cache (ARC) to store
both data and metadata. Metadata includes information like file paths, directory
structures, and permissions, which are frequently accessed in workloads involving
synchronous reads. By setting the cache to prioritize metadata
(primarycache=metadata), ZFS focuses on storing this critical information, ensuring
quicker lookups and file access. This avoids filling the cache with raw data blocks that
are less frequently reused, leading to higher cache efficiency and faster response
times.

Prefetching
Prefetching allows ZFS to anticipate sequential read operations by
loading adjacent blocks into memory before they are explicitly requested. While
synchronous reads can sometimes involve random I/O, many workloads (such as log
file processing) have a degree of sequential access. Keeping prefetching enabled
ensures that ZFS can proactively load contiguous data blocks into the ARC, reducing
disk reads and improving throughput for sequential parts of the workload. This
optimization is especially beneficial for workloads with mixed I/O patterns.

ARC Size Adjustment


The ARC is a memory-based cache that plays a critical role in
ZFS performance. By default, ZFS dynamically adjusts the ARC size based on available
system memory, but setting a hard limit (e.g., 50% of total memory) ensures that
other applications have sufficient resources while still allowing ZFS to cache
frequently accessed data and metadata effectively. This balance is crucial in systems
with limited memory, as it prevents ZFS from consuming excessive resources while
maintaining a performant cache.

You might also like