Learn How Alluxio 1.8.1 speeds up Alluxio metadata operations up to 100X

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Learn How Alluxio 1.8.1 speeds up Alluxio metadata operations up to 100X

binfan
Administrator
Hi, 

Alluxio core developer David Zhu recently posted an interesting blog on his optimization 
in Alluxio v1.8.1 that greatly improves some UFS-related Alluxio metadata operations by up to 100x,
including Alluxio-UFS sync operations and recursive list operations.

More work will be done in this direction in the next release, including:
  • Partition UFS Fingerprint between content-related vs metadata-related info (ALLUXIO-3150)
  • Use recursive listStatus in UFS to implement loadMetadata (ALLUXIO-3205)
  • Reduce the number of interactions with UFS in loadMetadata (ALLUXIO-3300)
You are welcome to contribute to Alluxio.


Click the following link to get more details and see if you can benefit from this optimization:
https://www.alluxio.com/blog/how-to-speed-up-alluxio-metadata-operations-up-to-100x

- Bin Fan

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.