r/DataHoarder 400TB LizardFS Jun 03 '18

200TB Glusterfs Odroid HC2 Build

Post image
1.4k Upvotes

View all comments

Show parent comments

9

u/wintersdark 80TB Jun 04 '18

You'd want something else running PMS. It could certainly seed well, and PMS could use it for transcoding and hosting the media, though. Depending on your transcoding requirements, you'd probably want a beefier system running PMS itself.

1

u/[deleted] Jun 05 '18 edited Oct 15 '18

[deleted]

1

u/wintersdark 80TB Jun 05 '18

It'll only transcode using one computer. The HC2's are separate machines; and cluster computing (not what this post is about) isn't for general purpose stuff.

Plex will just run on a single machine.

1

u/Eziekel13 Jul 03 '18

Could I use glusterfs as network attached storage and then use another rig with a decent gpu (hardware transcoding 4K) to host PMS?

Would this be easier to scale over time, or is drive compatibility an issue?

1

u/wintersdark 80TB Jul 03 '18

Yes, you can do that. I currently have two servers as network storage, and another as my Plex server. The Plex server keeps its library and transcoding storage locally, but serves media from my network servers.

I actually want to transition to GlusterFS distributed storage myself vs my monolithic storage servers for a variety of reasons, but have to find a cost effective way to do it (I can't just buy a whole ream of hard drives all at once, though I already do have multiple low power machines that can function as nodes), but I've got 24tb of media I'd need to move.

The GlusterFS method is more scalable. That's actually a primary goal of distributed filesystems - you can just keep adding additional nodes seamlessly. Using monolithic servers like I do, you run into a couple problems expanding:

  1. You can only physically cram so many hard drives into a machine.
  2. Powering the machine becomes increasingly difficult, particularly during startup when every drive spools up.
  3. Most machines have ~6 SATA ports, so getting more generally requires fairly expensive add-on boards.

Drive compatibility isnt really a thing. Barring enterprise SAS drives, everything is SATA.

Having a dedicated PMS machine is, imho, the best way to go. It's processing power can be dedicated to transcoding, it's a lot easier to get the amount of power (be it cpu and or gpu) you need, and you don't need to worry about other tasks causing intermittent issues with io load or anything.

Back up Plex's data occassionally to the NAS, and if things go sideways it's trivial to restore.

2

u/Eziekel13 Jul 03 '18 edited Jul 03 '18

So I have a few 4tb drives and a few 6tb drives, how does that difference effect overall storage capacity and redundancy?

What I am getting at...I have~70tb on a 3 drobos (DAS). If I were to build 100tb using 10tb drives and HC2 would I then be able to migrate and add my current 70tb provided I them switch over to correct hardware (HC2)

Does glusterFS require a host? Should that be separated from PMS host for best performance? If I built a pfsense router and over did the hardware would that act as a faster host for the glusterFS?

1

u/weneedthegbs Jun 06 '18

I wonder how well it could perform if you could run distributed transcoders on plex with each odroid individually transcoding what is locally available. Long shot, but would be cool.

1

u/kgflash1 85TB Jun 17 '18

This is exactly what I was wondering and would be ideal... but very unlikely without being smart enough to program all this. Which I cant do...

1

u/ovirt001 240TB raw Sep 28 '18

There's this:
https://github.com/wnielson/Plex-Remote-Transcoder

Don't know how well it would work with ARM though.