spark1.0.0 with alluxio1.0.0

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

spark1.0.0 with alluxio1.0.0

ge zhang

Hi guys,
I got a problem which I really can not solve. I establish a alluxio cluster on hdfs. Then I run spark1.0.0 on this cluster use this commond:
  1. val file = sc.textFile("/path/to/sample-1g")
  2. file.saveAsTextFile("alluxio://localhost:19998/sample-1g")
  3. val alluxioFile = sc.textFile("alluxio://localhost:19998/sample-1g")
  4. alluxioFile.count()
Then I got such problem

And I put the worker.log here. Can anyone help.


Best,

Ge

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

worker.log (17K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: spark1.0.0 with alluxio1.0.0

Pei Sun
Hi Ge,
     Thank you for trying Alluxio. This is a known issue. You can try our 1.1 version https://groups.google.com/forum/#!topic/alluxio-users/HWi8zeW3keI which should fix the problem. Let us know if you see any problem. 

Thanks
Pei

Pei

On Mon, May 23, 2016 at 4:55 AM, ge zhang <[hidden email]> wrote:

Hi guys,
I got a problem which I really can not solve. I establish a alluxio cluster on hdfs. Then I run spark1.0.0 on this cluster use this commond:
  1. val file = sc.textFile("/path/to/sample-1g")
  2. file.saveAsTextFile("alluxio://localhost:19998/sample-1g")
  3. val alluxioFile = sc.textFile("alluxio://localhost:19998/sample-1g")
  4. alluxioFile.count()
Then I got such problem

And I put the worker.log here. Can anyone help.


Best,

Ge

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.



--
Pei Sun

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.