Just started using SnapRAID (as part of OpenMediaVault and with mergerfs, though I am issuing commands at the terminal) and I noticed that doing a sync -h (sync with prehash) that the prehash stage only processes data at the speed of a single drive. Watching iostat -m confirms it is only reading from 1 drive at a time.
Unless there is a specific reason to do it this way, could it be changed to prehash from multiple drives at the same time (it is IO bound, with CPU usage being a mere 4%), or an option provided to enable it? Thanks!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Unless there is a specific reason to do it this way
Exactly!! pre-hash is a last-ditch workaround for a very obscure, and possibly unique, juxtapostition of hardware glitches, that would cause data errors; Unless you have been experiencing data errors, and have not been able to diagnose them via extensive use of "memtest" and Prime95-torture, you do not need/want to use snapraid -h.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Oh, I'm surprised to read that, as the manual makes no mention that it is for something so specific, only that "due the extreme conditions on the machine that sync creates, silent errors can occur and pre-hashing helps protect against them" (paraphrased).
So while reading from all the drives at the same time during pre-hashing would create those same extreme conditions, it would still offer protection from these silent errors (providing they were random).
In this light, leaving the existing option to only do 1 drive would be best, but adding a pre-hash that reads from all drives at the same time would still be useful.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I was just thinking about this more and realised I might not need that extra protection. I always scrub -p new after sync, so that would bring attention to any silent errors that occured during the sync.
Thanks UhClem for prompting me to look and think into this some more!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just started using SnapRAID (as part of OpenMediaVault and with mergerfs, though I am issuing commands at the terminal) and I noticed that doing a
sync -h(sync with prehash) that the prehash stage only processes data at the speed of a single drive. Watchingiostat -mconfirms it is only reading from 1 drive at a time.Unless there is a specific reason to do it this way, could it be changed to prehash from multiple drives at the same time (it is IO bound, with CPU usage being a mere 4%), or an option provided to enable it? Thanks!
Exactly!!
pre-hash is a last-ditch workaround for a very obscure, and possibly unique, juxtapostition of hardware glitches, that would cause data errors; Unless you have been experiencing data errors, and have not been able to diagnose them via extensive use of "memtest" and Prime95-torture, you do not need/want to use
snapraid -h.Oh, I'm surprised to read that, as the manual makes no mention that it is for something so specific, only that "due the extreme conditions on the machine that sync creates, silent errors can occur and pre-hashing helps protect against them" (paraphrased).
So while reading from all the drives at the same time during pre-hashing would create those same extreme conditions, it would still offer protection from these silent errors (providing they were random).
In this light, leaving the existing option to only do 1 drive would be best, but adding a pre-hash that reads from all drives at the same time would still be useful.
I was just thinking about this more and realised I might not need that extra protection. I always
scrub -p newaftersync, so that would bring attention to any silent errors that occured during thesync.Thanks UhClem for prompting me to look and think into this some more!