Is there auto fall back when exception happens during client read data from worker?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there auto fall back when exception happens during client read data from worker?

wayasxxx
Hi all,

I get a problem when using Spark SQL + Alluxio(1.8.0).
Sometimes Spark executor get timeout exception when it tries to read data from an Alluxio worker.
Is there a fall back mechanism in Alluxio?  I mean automatically fall back to UFS after failing several times to read data from alluxio worker.
If there is not, I think I need to handle this on client side, is that right?
Thanks for help.

Anyang


--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Is there auto fall back when exception happens during client read data from worker?

Lu Qiu
Hi Anyang,

We have different caching scenarios according to read data from Alluxio. The details are listed in https://www.alluxio.org/docs/master/en/Architecture-DataFlow.html.

If a file is stored in Alluxio and UFS, we will try to read the blocks of this file from alluxio workers. If we failed in reading one of the blocks, we will still try all Alluxio workers which have the block before we give up. we only retry certain types of exceptions (UnavailableException, ConnectException, DeadlineExceededException). 

After we make all efforts but still unable to provide the whole file to our users through Alluxio, we will automatically fall back to UFS mode (cache miss mode in our scenarios). The Alluxio client will delegate the read from UFS to a worker, preferably a local worker. This worker reads and caches the data from the under storage.

Thanks,
Lu




On Sun, Dec 2, 2018 at 7:18 PM <[hidden email]> wrote:
Hi all,

I get a problem when using Spark SQL + Alluxio(1.8.0).
Sometimes Spark executor get timeout exception when it tries to read data from an Alluxio worker.
Is there a fall back mechanism in Alluxio?  I mean automatically fall back to UFS after failing several times to read data from alluxio worker.
If there is not, I think I need to handle this on client side, is that right?
Thanks for help.

Anyang


--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.