alluxio.exception.BlockInfoException: Cannot complete a file without all the blocks committed

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

alluxio.exception.BlockInfoException: Cannot complete a file without all the blocks committed

Dima Fadeyev
Hello, everyone,

I'm trying to get started with alluxio. I'm having problems with copying file from my local filesystem to alluxio:

alluxio fs copyFromLocal file.txt /file.txt
alluxio.exception.BlockInfoException: Cannot complete a file without all the blocks committed

Any file I try to copy ends up with the same error. 

The file becomes visible in alluxio anyway:

alluxio fs ls /
0.00B     05-24-2016 10:56:22:008  In Memory      /file.txt

However it always has 0 bytes. If I try to cat the output, there is none.

I'm testing alluxio on a single machine. I've downloaded and unpacked alluxio-1.0.1-bin.tar.gz. The only variables I've changed in alluxio-env.sh are:
JAVA_HOME=/opt/java
ALLUXIO_MASTER_ADDRESS=master.local
ALLUXIO_UNDERFS_ADDRESS=/tmp/alluxio
ALLUXIO_WORKER_MEMORY_SIZE=256mb

Then I've started alluxio like this:

alluxio format
/opt/alluxio/bin/alluxio-start.sh master
/opt/alluxio/bin/alluxio-start.sh worker Mount

Have I missed something?

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: alluxio.exception.BlockInfoException: Cannot complete a file without all the blocks committed

Gene Pang
Hi,

Could you show some of the logs? Specifically, could you see the logs for the application, and the worker? The logs should be found in in your path "/opt/alluxio/logs".

Also, have you tried increasing the "ALLUXIO_WORKER_MEMORY_SIZE"? This might be too small, since I think the default block size is 500MB.

Thanks,
Gene

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: alluxio.exception.BlockInfoException: Cannot complete a file without all the blocks committed

Dima Fadeyev
Hello, Gene,

Thanks a lot for your answer. You are right. 256mb was too small. After setting ALLUXIO_WORKER_MEMORY_SIZE to 1gb copyFromLocal is working.

Thanks once again and best regards.
Dima Fadeyev

On Tuesday, May 24, 2016 at 5:14:02 PM UTC+2, Gene Pang wrote:
Hi,

Could you show some of the logs? Specifically, could you see the logs for the application, and the worker? The logs should be found in in your path "/opt/alluxio/logs".

Also, have you tried increasing the "ALLUXIO_WORKER_MEMORY_SIZE"? This might be too small, since I think the default block size is 500MB.

Thanks,
Gene

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: alluxio.exception.BlockInfoException: Cannot complete a file without all the blocks committed

Gene Pang
Hi Dima,

I'm glad you got the problem resolved, and thanks for confirming!

-Gene

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.