Range reading on S3 object

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Range reading on S3 object

francois.autaa.pro
Hi all me again ...

Alluxio 1.7/1.8
Centos
Boto3



I would like, for project reason, to read only part of an s3 object (without downloading full object and after getting only the part I want).

Following this doc => https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html the range will do what I want.

I'm doing it in Python with boto3 so https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.get_object the boto3 Client .get_Object have a Range param but it doesn't work at all.

I've tried to acess it usinf curl by adding -r 1-40 or -H "Range:bytes=1-40" none of them give me expected result I every time get my full file ...

Is the "range" notion supported by Alluxio ?

Regards

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Range reading on S3 object

Calvin Jia
Hi,

Currently Alluxio's S3 API does not support range queries. If you open a JIRA ticket detailing the expected behavior and your intended use cases that would be helpful as we plan to implement this in the future.

Hope this helps,
Calvin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Range reading on S3 object

francois.autaa.pro


Done with the JIRA => https://alluxio.atlassian.net/browse/ALLUXIO-3321

Please ask any required information I've missed on the ticket.

Le mercredi 12 septembre 2018 20:06:56 UTC+2, Calvin Jia a écrit :
Hi,

Currently Alluxio's S3 API does not support range queries. If you open a <a href="https://alluxio.atlassian.net/projects/ALLUXIO/issues" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Falluxio.atlassian.net%2Fprojects%2FALLUXIO%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGTFif7qvZoNAXjhkcosrkzwqPyng&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Falluxio.atlassian.net%2Fprojects%2FALLUXIO%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGTFif7qvZoNAXjhkcosrkzwqPyng&#39;;return true;">JIRA ticket detailing the expected behavior and your intended use cases that would be helpful as we plan to implement this in the future.

Hope this helps,
Calvin

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.