Reading update drecords

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Reading update drecords

RUTVIK BHOPATRAO
I am testing Alluxio for reading data from AWS S3 to be read for other uses. But every time I update the records for the file, the data read won't change. It shows me the previously persisted data. How do I view the updated recods everytime i read the file?

Alluxio 1.8.0
Java 1.8.0
Storage: AWS s3
reading into AWS EC2 Linux

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Reading update drecords

Andrew Audibert
Hi Rutvik,

With default configuration, Alluxio expects UFS (S3 in your case) modifications to happen via Alluxio, so it doesn't need to constantly re-scan the UFS to look for updates. To make Alluxio aware of UFS updates, you can set alluxio.user.file.metadata.sync.interval=0. See this page for a full discussion of how to make Alluxio aware of UFS updates: https://www.alluxio.org/docs/master/en/Loading-Under-File-Storage-Metadata.html

Hope that helps,
Andrew

On Thu, Oct 18, 2018 at 9:04 AM RUTVIK BHOPATRAO <[hidden email]> wrote:
I am testing Alluxio for reading data from AWS S3 to be read for other uses. But every time I update the records for the file, the data read won't change. It shows me the previously persisted data. How do I view the updated recods everytime i read the file?

Alluxio 1.8.0
Java 1.8.0
Storage: AWS s3
reading into AWS EC2 Linux

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
--

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.