Why alluxio 1.8.0 deployment need hostPID setup ?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Why alluxio 1.8.0 deployment need hostPID setup ?

Kun Li
Hi, guys

I'm now deploying alluxio 1.8.0 into my k8s cluster. I noticed that alluxio deployments have 'hostPID: true' setup, is this setup neccesary ?

likun

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Why alluxio 1.8.0 deployment need hostPID setup ?

adit
Hi,

`hostPID:true` is used for locality scheduling. Locality improves performance for the short circuit access path. Which compute framework and version are you using?

Adit 

On Tuesday, September 11, 2018 at 7:40:12 PM UTC-7, Kun Li wrote:
Hi, guys

I'm now deploying alluxio 1.8.0 into my k8s cluster. I noticed that alluxio deployments have 'hostPID: true' setup, is this setup neccesary ?

likun

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Why alluxio 1.8.0 deployment need hostPID setup ?

Kun Li
Thanks Adit.  I'm running alluxio 1.8.0 with spark 2.3.1.

On Thursday, September 13, 2018 at 6:55:29 AM UTC+8, [hidden email] wrote:
Hi,

`hostPID:true` is used for locality scheduling. Locality improves performance for the short circuit access path. Which compute framework and version are you using?

Adit 

On Tuesday, September 11, 2018 at 7:40:12 PM UTC-7, Kun Li wrote:
Hi, guys

I'm now deploying alluxio 1.8.0 into my k8s cluster. I noticed that alluxio deployments have 'hostPID: true' setup, is this setup neccesary ?

likun

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Why alluxio 1.8.0 deployment need hostPID setup ?

adit
Thanks Kun. Short circuit access does not work w/o Spark kubernetes volumes. That feature is not available with Spark 2.3.1 and is available only with Spark 2.4.0. Ref: https://github.com/apache/spark/blob/master/docs/running-on-kubernetes.md (Using Kubernetes Volumes). The hostPath volume should then be configured with the alluxio domain socket directory. 

Adit

On Thursday, September 13, 2018 at 6:13:11 PM UTC-7, Kun Li wrote:
Thanks Adit.  I'm running alluxio 1.8.0 with spark 2.3.1.

On Thursday, September 13, 2018 at 6:55:29 AM UTC+8, [hidden email] wrote:
Hi,

`hostPID:true` is used for locality scheduling. Locality improves performance for the short circuit access path. Which compute framework and version are you using?

Adit 

On Tuesday, September 11, 2018 at 7:40:12 PM UTC-7, Kun Li wrote:
Hi, guys

I'm now deploying alluxio 1.8.0 into my k8s cluster. I noticed that alluxio deployments have 'hostPID: true' setup, is this setup neccesary ?

likun

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.