Troubleshooting

Gathering information

RozoFS daemons write to syslog. Read your main syslog first.

If you use the manager CLI, you can see daemons status with:

# rozo status --exportd <ip addr of exportd node>

Performance

Disk performance

Raw FS can be tuned by changing the IP scheduler:

Install sysfsutils:

apt-get install sysfsutils

Configure :

# /etc/sysfs.conf - Configuration file for setting sysfs attributes.
block/sda/queue/scheduler = deadline
block/sda/queue/iosched/front_merges = 0
block/sda/queue/iosched/read_expire = 150
block/sda/queue/iosched/write_expire = 1500

Network performance

RozoFS is network intensive. It can produce congestion on ethernet switches. This can be seen with dropped packets on the path.

Possible ways to fix congestion (none are mandatory):

• Increase the bandwith (using 10G ethernet)
• Increase the bandwith using bonding (check that your traffic is well distributed, or change the hash algorithm or IP addresses accordingly)
• Enable FlowControl
• Enable ECN
• Upgrate Linux kernel, with incoming Bufferbloat fixes:
• Linux 3.3: Bufferbloat fighting: Byte queue limits
• Linux 3.5: Bufferbloat fighting: CoDel queue management
• Linux 3.5: TCP Early Retransmit
• Linux 3.6: Bufferbloat fight: TCP small queues
• Linux 3.10: TCP optimization: Tail loss probe
• Reduce bufferbloat (with something like: https://github.com/dtaht/deBloat/blob/master/debianstuff/if-pre-up.d/debloat)
• Reduce the number of hops and avoid routing
• Tune tso and gso, lro, ...
• Enable traffic shaping in RozoFS (experimental feature)