can alluxio bring block redundancy?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

can alluxio bring block redundancy?

wgcris
when i run MR on alluxio twice, from the webUI, i found the same block is stored in different worker node, i have a question whether this case can bring data redundancy?  anyone can make a explanantion? thanks

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

fig4.PNG (107K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: can alluxio bring block redundancy?

Calvin Jia
Hi,

Alluxio will load blocks into the local worker's memory as data is read on that node. What you are seeing is most likely due to the mappers running on different nodes between the first and second run, or if this happens on the first run, the mapper splits are not well aligned with the blocks.

Could you share what kind of workload you are running and some more details about your setup, ie. Alluxio version?

Hope this helps,
Calvin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: can alluxio bring block redundancy?

Calvin Jia
Hi,

Were you able to find the cause of the behavior?

Thanks,
Calvin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.