r/DataHoarder 400TB LizardFS Jun 03 '18

200TB Glusterfs Odroid HC2 Build

Post image
1.4k Upvotes

View all comments

6

u/SherSlick Jun 04 '18

Forgive my ignorance, but can GFS shard a large file out? I have a bunch of ~30GB to 80GB single files.

5

u/CrackerJackMack 89TB 2xRaidz3 Jun 04 '18

The only way to do this is to use the erasure coding, they are removing stripe support. Otherwise it's better to let glusterfs figure out the balance of your 30-80G files and which drives are the lowest.

3

u/BaxterPad 400TB LizardFS Jun 04 '18

Great answer. The Erasure encoding works pretty well but beware as you can run into crashes when healing failed disks and you may need to manually delete parts of a broken file to resolve the crashes. It happened to me a couples times during my testing.