Alluxio V1.8.1 : how to set glusterfs as underfs when GlusterFS is not co-located with Alluxio node

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Alluxio V1.8.1 : how to set glusterfs as underfs when GlusterFS is not co-located with Alluxio node

mianguanwu

Hi,
     We use three machines to deploy Alluxio 1.8.1 release.  now we want to configure Alluxio with GlusterFS as the under storage system.  we have checked https://www.alluxio.org/docs/1.8/en/ufs/GlusterFS.html. but we cannot find how to configure if GlusterFS is not  co-located with Alluxio nodes.   we configure alluxio-site.properties as follows:

165470934.jpg

    But after  run "./bin/alluxio-start.sh all". we cannot found the UnderFS capacity in website "master:19999/home".  we check the master.log ,we found alluxio cluster cannot found glusterfs :

捕获.PNG

    

      If GlusterFS bricks are co-located with Alluxio nodes, we configure alluxio-site.properties with "alluxio.underfs.address =/mnt", alluxio can find GlusterFS bricks.  Now,  we are wondering how to configure Glusterfs as underfs when the  GlusterFS bricks is not  co-located with Alluxio nodes.
Another question is how to make sure the all the data write to alluxio will persist in underfs like GlusterFS .

Thanks 
Kevin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Alluxio V1.8.1 : how to set glusterfs as underfs when GlusterFS is not co-located with Alluxio node

Bin Fan
hi Kevin,

for the first question, right now Alluxio only supports to connect to GlusterFS through local mount (co-located with Alluxio nodes).
Alluxio does not have a connector to connect to remote GlusterFS currently.
Here are tutorial to develop such a connector (link)

for the second question, overwrite property key
alluxio.user.file.writetype.default=CACHE_THROUGH
to enable sync write to Alluxio & UFS.

check here https://www.alluxio.org/docs/1.8/en/basic/Configuration-Settings.html#alluxio-shell-commands
for more details.

- Bin

On Wednesday, October 31, 2018 at 7:37:14 PM UTC-7, [hidden email] wrote:

Hi,
     We use three machines to deploy Alluxio 1.8.1 release.  now we want to configure Alluxio with <a href="http://www.gluster.org/" style="background-color:rgb(247,248,250);color:rgb(21,111,219);font-family:&quot;Source Sans Pro&quot;,&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.gluster.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFca7emY9gdfVFLmrHZQmMVXpJ24Q&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.gluster.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFca7emY9gdfVFLmrHZQmMVXpJ24Q&#39;;return true;">GlusterFS as the under storage system.  we have checked <a href="https://www.google.com/url?q=https%3A%2F%2Fwww.alluxio.org%2Fdocs%2F1.8%2Fen%2Fufs%2FGlusterFS.html&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNGk-wvC-uZr18V6bzvQcQeLFiY6Dg" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.alluxio.org%2Fdocs%2F1.8%2Fen%2Fufs%2FGlusterFS.html\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGk-wvC-uZr18V6bzvQcQeLFiY6Dg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.alluxio.org%2Fdocs%2F1.8%2Fen%2Fufs%2FGlusterFS.html\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGk-wvC-uZr18V6bzvQcQeLFiY6Dg&#39;;return true;">https://www.alluxio.org/docs/1.8/en/ufs/GlusterFS.html. but we cannot find how to configure if <a href="http://www.gluster.org/" style="color:rgb(21,111,219);background-color:rgb(247,248,250);font-family:&quot;Source Sans Pro&quot;,&quot;Helvetica Neue&quot;,Helvetica,Arial,sans-serif" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.gluster.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFca7emY9gdfVFLmrHZQmMVXpJ24Q&#39;;return true;" onclick="this.href=&#39;http://www.google.com/url?q\x3dhttp%3A%2F%2Fwww.gluster.org%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFca7emY9gdfVFLmrHZQmMVXpJ24Q&#39;;return true;">GlusterFS is not  co-located with Alluxio nodes.   we configure alluxio-site.properties as follows:

165470934.jpg

    But after  run "./bin/alluxio-start.sh all". we cannot found the UnderFS capacity in website "master:19999/home".  we check the master.log ,we found alluxio cluster cannot found glusterfs :

捕获.PNG

    

      If GlusterFS bricks are co-located with Alluxio nodes, we configure alluxio-site.properties with "alluxio.underfs.address =/mnt", alluxio can find GlusterFS bricks.  Now,  we are wondering how to configure Glusterfs as underfs when the  GlusterFS bricks is not  co-located with Alluxio nodes.
Another question is how to make sure the all the data write to alluxio will persist in underfs like GlusterFS .

Thanks 
Kevin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.