Tier Stoarage Not Coming in the UI

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Tier Stoarage Not Coming in the UI

Pranjul Ahuja
Hi,

I have configured tiered storage in my site properties but it is not reflecting in the UI.

Here is the configs : 

alluxio.worker.memory.size=20GB
alluxio.master.tieredstore.global.levels=2
alluxio.master.tieredstore.global.level0.alias=MEM
alluxio.master.tieredstore.global.level1.alias=HDD
alluxio.worker.tieredstore.levels=2
alluxio.worker.tieredstore.level0.alias=MEM
alluxio.worker.tieredstore.level0.dirs.path=/mnt/ramdisk
alluxio.worker.tieredstore.level0.dirs.quota=20GB
alluxio.worker.tieredstore.level0.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level0.watermark.low.ratio=0.7
alluxio.worker.tieredstore.level1.alias=HDD
alluxio.worker.tieredstore.level1.dirs.path=/
alluxio.worker.tieredstore.level1.dirs.quota=100GB
alluxio.worker.tieredstore.level1.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level1.watermark.low.ratio=0.7

I am using sudoMount while starting the workers. Is there anything i am missing ? Moreover, i have set memory storage to 20GB but in the UI, it shows workers capacity to be 39.32GB.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Tier Stoarage Not Coming in the UI

Bin Fan
the configuration seems correct--- assume this configuration is made in ${ALLUXIO_HOME}/conf/alluxio-site.properties
- did you restart your masters to pick up the latest change?
- "alluxio.worker.tieredstore.level1.dirs.path=/" not sure if this is the real setting. if so, this may have issues on root
- if you are running 1.8.x, can you run the following to get information ?


$ bin/alluxio fsadmin report
Alluxio cluster summary: 
    Master Address: localhost/127.0.0.1:19998
    Web Port: 19999
    Rpc Port: 19998
    Started: 10-30-2018 13:44:21:349
    Uptime: 0 day(s), 0 hour(s), 0 minute(s), and 25 second(s)
    Version: 2.0.0-SNAPSHOT
    Safe Mode: true
    Zookeeper Enabled: false
    Live Workers: 1
    Lost Workers: 0
    Total Capacity: 12.00GB
        Tier: MEM  Size: 2048.00MB
        Tier: HDD  Size: 10.00GB
    Used Capacity: 0B
        Tier: MEM  Size: 0B
        Tier: HDD  Size: 0B
    Free Capacity: 12.00GB

On Tue, Oct 30, 2018 at 2:10 AM Pranjul Ahuja <[hidden email]> wrote:
Hi,

I have configured tiered storage in my site properties but it is not reflecting in the UI.

Here is the configs : 

alluxio.worker.memory.size=20GB
alluxio.master.tieredstore.global.levels=2
alluxio.master.tieredstore.global.level0.alias=MEM
alluxio.master.tieredstore.global.level1.alias=HDD
alluxio.worker.tieredstore.levels=2
alluxio.worker.tieredstore.level0.alias=MEM
alluxio.worker.tieredstore.level0.dirs.path=/mnt/ramdisk
alluxio.worker.tieredstore.level0.dirs.quota=20GB
alluxio.worker.tieredstore.level0.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level0.watermark.low.ratio=0.7
alluxio.worker.tieredstore.level1.alias=HDD
alluxio.worker.tieredstore.level1.dirs.path=/
alluxio.worker.tieredstore.level1.dirs.quota=100GB
alluxio.worker.tieredstore.level1.watermark.high.ratio=0.9
alluxio.worker.tieredstore.level1.watermark.low.ratio=0.7

I am using sudoMount while starting the workers. Is there anything i am missing ? Moreover, i have set memory storage to 20GB but in the UI, it shows workers capacity to be 39.32GB.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Tier Stoarage Not Coming in the UI

Pranjul Ahuja
Hi Bin,
I ran the above mentioned command and only MEM tier is coming in the output. I am running alluxio only after updating the properties file so I guess there should be no need to restart.
Can we give any directory path in  "alluxio.worker.tieredstore.level1.dirs.path=/" ?

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Tier Stoarage Not Coming in the UI

Bin Fan
Hi, Pranjul,

if only MEM tier is shown in the output of "fsadmin report" command,
that means the two-tier setting is not taking effect. For the original configuration 
you put in the original post, how did you set it?

You can supply any writable dir to "alluxio.worker.tieredstore.level1.dirs.path=", e.g.
it is typically like:
"alluxio.worker.tieredstore.level1.dirs.path=/tmp/alluxio/"

A few things to help you check out:

- how many masters are you running? if there are multiple masters, you should
update the conf/alluxio-site.properties file for each master instance

- You can run "bin/alluxio getConf --master" to check the active configuration
on the master side

- See how Alluxio server-side configuration works: 
https://www.alluxio.org/docs/1.8/en/basic/Configuration-Settings.html#configure-alluxio-cluster

Hope they help.

- Bin

On Tuesday, October 30, 2018 at 8:28:26 PM UTC-7, Pranjul Ahuja wrote:
Hi Bin,
I ran the above mentioned command and only MEM tier is coming in the output. I am running alluxio only after updating the properties file so I guess there should be no need to restart.
Can we give any directory path in  "alluxio.worker.tieredstore.level1.dirs.path=/" ?

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Tier Stoarage Not Coming in the UI

Pranjul Ahuja
Hi Bin,

It got resolved. User id  from which i invoked the worker processes did not have permission to write into the configured directory.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Tier Stoarage Not Coming in the UI

Bin Fan
great! glad it got resolved!

- Bin

On Mon, Nov 5, 2018 at 2:06 AM Pranjul Ahuja <[hidden email]> wrote:
Hi Bin,

It got resolved. User id  from which i invoked the worker processes did not have permission to write into the configured directory.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.