Alluxio 1.8 is not writing file completely

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Alluxio 1.8 is not writing file completely

Jais Sebastian
We are getting inputstream from JSCH ( Java SFTP ) and using Alluxio Hadoop file system abstraction APIs we are saving into Alluxio. But now the file is not getting saved completely. We see a zero size file getting created in Alluxio. 
This is working fine if we use Alluxio 1.5 server. Is there any specific configuration needs to be updated here? 

Regards,
Jais

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Alluxio 1.8 is not writing file completely

Bin Fan
hi Jais,

1. does this happen deterministically?
2. can you check Alluxio master and worker logs and see if there are any exceptions thrown?
3. on the client side, do you see any exceptions or any logs at all indicating the interaction with Alluxio at all?

- Bin


On Wed, Sep 19, 2018 at 7:43 AM Jais Sebastian <[hidden email]> wrote:
We are getting inputstream from JSCH ( Java SFTP ) and using Alluxio Hadoop file system abstraction APIs we are saving into Alluxio. But now the file is not getting saved completely. We see a zero size file getting created in Alluxio. 
This is working fine if we use Alluxio 1.5 server. Is there any specific configuration needs to be updated here? 

Regards,
Jais

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Alluxio 1.8 is not writing file completely

Jais Sebastian
We could resolve this issue after upgrading our client application netty library to latest version.
Yes, this issue was happening consistently.

On Thursday, September 20, 2018 at 5:03:22 AM UTC+5:30, Bin Fan wrote:
hi Jais,

1. does this happen deterministically?
2. can you check Alluxio master and worker logs and see if there are any exceptions thrown?
3. on the client side, do you see any exceptions or any logs at all indicating the interaction with Alluxio at all?

- Bin


On Wed, Sep 19, 2018 at 7:43 AM Jais Sebastian <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="8iMA0ApoCgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jais...@...> wrote:
We are getting inputstream from JSCH ( Java SFTP ) and using Alluxio Hadoop file system abstraction APIs we are saving into Alluxio. But now the file is not getting saved completely. We see a zero size file getting created in Alluxio. 
This is working fine if we use Alluxio 1.5 server. Is there any specific configuration needs to be updated here? 

Regards,
Jais

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="8iMA0ApoCgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">alluxio-user...@googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.