Alluxio cluster with local storage

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Alluxio cluster with local storage

Bill Graham
Hi,

I'm prototyping using Alluxio as a cache between Presto and S3. I'm colocating the Alluxio processes on the presto nodes and looking to use local storage as the cache. Reading the docs this doesn't seem possible. Is there any way to use local storage for the cluster cache? I'm trying to cache S3 calls so using S3 (or some other remote store) as the under storage system defeats the purpose.

thanks,
Bill

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Alluxio cluster with local storage

Bill Graham
Answering my own question, what I want to do is indeed possible. I was mixing up terminology of what is the under storage system here (https://www.alluxio.org/docs/1.7/en/Running-Alluxio-on-a-Cluster.html):

You cannot use local file system as Alluxio’s under storage system if there are multiple nodes in the cluster. Instead you need to set up a shared storage to which all Alluxio servers have access. 

Under storage is the thing I'm caching, not the cache. This page cleared things up:

On Wed, Jul 25, 2018 at 1:51 PM Bill Graham <[hidden email]> wrote:
Hi,

I'm prototyping using Alluxio as a cache between Presto and S3. I'm colocating the Alluxio processes on the presto nodes and looking to use local storage as the cache. Reading the docs this doesn't seem possible. Is there any way to use local storage for the cluster cache? I'm trying to cache S3 calls so using S3 (or some other remote store) as the under storage system defeats the purpose.

thanks,
Bill

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.