Re: How data blocks are partitioned

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: How data blocks are partitioned

Andrew Audibert
Hi pedram,

Alluxio uses location policies to decide how to partition blocks. By default Alluxio will write to the local worker if there is one, otherwise it will choose a random worker. The copyFromLocal shell command is an exception; by default it will partition the file's blocks evenly across all workers.

-Andrew

On Thu, Nov 15, 2018 at 2:52 PM ems eril <[hidden email]> wrote:
Hi Team ,

    I have simple question about how data is partitioned . If we have 2 nodes each with 80 GB total RAM and we allocate 50GB for each node what happens when you try to store a file that is 60GB or more ? How does partitioning rule work in Alluxio ?

Thanks

-pedram

--
You received this message because you are subscribed to the Google Groups "Alluxio Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
--

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.