Alluxio fs persist corrupted

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Alluxio fs persist corrupted

ahguang973
I am developing my own under fs to alluxio, like the one s3a, however, when I persist the data , I get strange in master:

I print out the log for listObject and getObjectMetadata, and 
From below log, we can see that: alluxio framework send some strong key to me like 
.hotspotrc
org
java
com
io
android
javassist

sounds like a package name, and something is corrupted inside alluxio framework.
the dir1 is my directory name "/dir1", I don't have anything like  org.java.com.io.android.javassist
sounds like a memory corruption. Maybe my code is not stable yet, but shouldn't it impact the alluxio original code?
 I am using 1.7

BTW, I didn't touch any of alluxio code, and my code is built into a jar and feed into alluxio.


2018-03-24 13:07:38,849 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc

2018-03-24 13:07:38,935 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc_$folder$

2018-03-24 13:07:39,005 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspotrc/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,164 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,177 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc

2018-03-24 13:07:39,238 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler

2018-03-24 13:07:39,241 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc_$folder$

2018-03-24 13:07:39,292 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler_$folder$

2018-03-24 13:07:39,306 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspotrc/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,352 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspot_compiler/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,456 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,491 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,505 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler

2018-03-24 13:07:39,568 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler_$folder$

2018-03-24 13:07:39,598 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:39,659 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:39,724 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,858 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,868 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:39,929 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:39,933 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:39,986 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:39,991 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,044 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,105 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,107 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,112 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:40,130 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,191 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,192 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:40,246 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,255 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,369 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,372 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspot_compiler/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,376 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,379 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,380 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,437 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,441 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,493 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,513 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,531 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,610 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,611 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,616 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,690 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,750 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,861 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,873 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:40,937 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,035 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,163 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,170 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:41,182 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:41,253 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,258 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,337 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,343 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,477 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,477 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,482 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:41,571 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,632 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,648 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:41,702 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:41,730 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,764 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,867 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,871 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:41,878 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:41,933 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:41,936 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:41,990 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,993 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,090 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,093 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,099 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:42,154 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:42,213 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,326 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:42,395 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:07:42,450 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,545 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,552 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,561 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:42,562 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:42,626 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:07:42,630 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:07:42,691 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,694 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:52,806 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:52,806 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:52,822 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:52,971 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android

2018-03-24 13:08:02,881 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:08:02,941 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:03,043 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android_$folder$

2018-03-24 13:08:03,101 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/android/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:03,750 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:03,968 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:03,975 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android

2018-03-24 13:08:03,988 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android

2018-03-24 13:08:04,035 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android_$folder$

2018-03-24 13:08:04,045 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android_$folder$

2018-03-24 13:08:04,086 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/android/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:04,102 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/android/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:04,591 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:04,598 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:04,645 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF

2018-03-24 13:08:04,703 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF_$folder$

2018-03-24 13:08:04,762 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/META-INF/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:05,251 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:05,254 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF

2018-03-24 13:08:05,318 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF_$folder$

2018-03-24 13:08:05,383 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/META-INF/ marker:null maxkeys:1000 delimiter:


--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Alluxio fs persist corrupted

adit
Hi,

Can you please share:
1. ufs.address configuration
2. a listing of the contents of the above address

Also, some questions:
- does mount succeed?
- what are the contents of the under store after the persist operation?
- is there an error message on the alluxio client or server?

To help debug, you may want to use the debugging options listed here to attach the your IDE to the Alluxio master process and step through the ObjectUnderFileSystem methods.

Thanks,

Adit

On Saturday, March 24, 2018 at 6:19:26 AM UTC-7, [hidden email] wrote:
I am developing my own under fs to alluxio, like the one s3a, however, when I persist the data , I get strange in master:

I print out the log for listObject and getObjectMetadata, and 
From below log, we can see that: alluxio framework send some strong key to me like 
.hotspotrc
org
java
com
io
android
javassist

sounds like a package name, and something is corrupted inside alluxio framework.
the dir1 is my directory name "/dir1", I don't have anything like  org.java.com.io.android.javassist
sounds like a memory corruption. Maybe my code is not stable yet, but shouldn't it impact the alluxio original code?
 I am using 1.7

BTW, I didn't touch any of alluxio code, and my code is built into a jar and feed into alluxio.


2018-03-24 13:07:38,849 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc

2018-03-24 13:07:38,935 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc_$folder$

2018-03-24 13:07:39,005 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspotrc/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,164 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,177 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc

2018-03-24 13:07:39,238 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler

2018-03-24 13:07:39,241 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspotrc_$folder$

2018-03-24 13:07:39,292 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler_$folder$

2018-03-24 13:07:39,306 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspotrc/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,352 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspot_compiler/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,456 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,491 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,505 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler

2018-03-24 13:07:39,568 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/.hotspot_compiler_$folder$

2018-03-24 13:07:39,598 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:39,659 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:39,724 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:39,858 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:39,868 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:39,929 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:39,933 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:39,986 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:39,991 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,044 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,105 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,107 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,112 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio

2018-03-24 13:07:40,130 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,191 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,192 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/alluxio_$folder$

2018-03-24 13:07:40,246 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/alluxio/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,255 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,369 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,372 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/.hotspot_compiler/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,376 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,379 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,380 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,437 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,441 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,493 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,513 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,531 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,610 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,611 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,616 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org

2018-03-24 13:07:40,690 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/org_$folder$

2018-03-24 13:07:40,750 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/org/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:40,861 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:40,873 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:40,937 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,035 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,163 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,170 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:41,182 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:41,253 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,258 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,337 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,343 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,477 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,477 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,482 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java

2018-03-24 13:07:41,571 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/java_$folder$

2018-03-24 13:07:41,632 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/java/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,648 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:41,702 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:41,730 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,764 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,867 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:41,871 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:41,878 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:41,933 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:41,936 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:41,990 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:41,993 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,090 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,093 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,099 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com

2018-03-24 13:07:42,154 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/com_$folder$

2018-03-24 13:07:42,213 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/com/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,326 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:42,395 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:07:42,450 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,545 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,552 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:42,561 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:42,562 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:42,626 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:07:42,630 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:07:42,691 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:42,694 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:07:52,806 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:52,806 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:07:52,822 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io

2018-03-24 13:07:52,971 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android

2018-03-24 13:08:02,881 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/io_$folder$

2018-03-24 13:08:02,941 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/io/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:03,043 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android_$folder$

2018-03-24 13:08:03,101 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/android/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:03,750 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:03,968 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:03,975 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android

2018-03-24 13:08:03,988 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android

2018-03-24 13:08:04,035 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android_$folder$

2018-03-24 13:08:04,045 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/android_$folder$

2018-03-24 13:08:04,086 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/android/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:04,102 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/android/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:04,591 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:04,598 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:04,645 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF

2018-03-24 13:08:04,703 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF_$folder$

2018-03-24 13:08:04,762 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/META-INF/ marker:null maxkeys:1000 delimiter:

2018-03-24 13:08:05,251 WARN  oss - ==== listObject count:0 marker:null EOF:true prefixes:

2018-03-24 13:08:05,254 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF

2018-03-24 13:08:05,318 WARN  oss - ==== getObjectMetadata bucket:lichg key:dir1/META-INF_$folder$

2018-03-24 13:08:05,383 WARN  oss - ==== listObject headers:{} params:{} encode:null bucket:lichg key:null prefix:dir1/META-INF/ marker:null maxkeys:1000 delimiter:


--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Alluxio fs persist corrupted

Calvin Jia
In reply to this post by ahguang973
Hi,

What was the persist error you got?

For the strange list status / get object metadata, could you verify if your under store is configured correctly? For example, it could be returning files which are from your local file system?

Hope this helps,
Calvin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.