Putting the btrfs / zfs requirement aside - in general you'll have no problem.
I'm running a group that's nearly all RPi - with the main 'master' source running on an Odroid XU4 (you will get away with a USB3 drive there too).
Obviously don't expect blazing speeds but it will work. For best results, ensure the RPi are disconnected rarely, and at the time they are reconnected it's best to leave them so at least overnight.
The Odroid XU4 won't become corrupted.
Maybe once / twice a year - you'll decide to wipe the index on an RPi and let it rebuild.
Other tips are:
Spread out folders. Don't have 'Photos' for all Photos, instead have Photos-2016-P1, Photos-2016-P2, etc. and so on for Videos, Audio and whatever data can be meaningfully broken down into discrete folders. This will reduce the hashing & thrashing.
Always quit the Pi gracefully, if you're making this for someone else - really emphasise this. Plugging a Pi out from the wall suddenly will eventually require a visit from someone with know-how.
To branch out from the Pi, but stay in the same price range - there are the Odroid C2.
You can add an expense here with eMMC cards not only SDs. This makes the C2 practically immune to the SD corruption possible on the Pi, and noticeably faster.
From a media-centre point of view the C2 are 4K ready with HEVC - so beats the Pi squarely on that front.
I've been running this setup for over 18 months now.
Very doable, and not as painful as many would suggest.