<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Hello everyone,</div><div class=""><br class=""></div><div class="">I would like to share one of the topics about the event I participated — AWS </div><div class="">Optimizing Storage Storage for Big Data/Analytics Workloads.</div><div class=""><br class=""></div><div class="">As you may know, AWS has a series of products used to deal with data for </div><div class="">different purposes (EMR for data computing, Redshift for data warehouse, </div><div class="">Kinesis for streaming, DynamoDB for NoSQL, ElastiSearch for searching, etc.). </div><div class="">Thus, the way to store the data should be taken into consideration for better </div><div class="">performance. AWS also provides a couple of storage ways: S3 (Multi-tenant </div><div class="">Key-store Native API), EFS (Shared/Distributed POSIX NFS/SMB), EBS </div><div class="">(Single-tenant Block-store). </div><div class=""><br class=""></div><div class="">For this lecture, we are focusing on EBS(Elastic Block Store). </div><div class=""><br class=""></div><div class="">Basically, EBS provides 4 volume types: gp2, io1, st1 and sc1. Choosing </div><div class="">volume type depends on what is your expectation — which is more important: </div><div class="">IOPS(Input/output operations per second) or Throughput. If you look for better </div><div class="">IOPS, then SSD-based volumes are good choices for you (gp2, io1). However, if </div><div class="">you need more important throughput, then HDD-based volumes are the ones</div><div class="">(sc1, st1). Of course, there are more options for you if you look for more </div><div class="">accurate performance. For example extremely high IOPS or low latency less </div><div class="">than 1ms, you should consider i2 rather than EBS type(Slides below may be </div><div class="">more intuitive).</div><div class=""><br class=""></div><div class="">So far, we are clear about what AWS stores data with its products: S3 is for Big </div><div class="">Data&Analytics, and EBS is used for Data Warehouses, Search& Indexing, </div><div class="">Transactional& NoSQL Databases and Streaming data. And EBS will be finally </div><div class="">managed with S3.</div><div class=""><br class=""></div><div class="">More interesting news about this topic is EBS Elastic Volumes, a new feature in </div><div class="">2017. Elastic Volumes is used to modify the configuration of live volumes </div><div class="">attached to instances in a simple, flexible, non-disruptive and automated way. </div><div class="">The idea here is that with this feature we can dynamically increase size, tune </div><div class="">performance and change the type of existing and new current generation </div><div class="">volumes. However, there are also some limitations for current features. For </div><div class="">example, you can modify a volume once per 6 hours, it supported only for </div><div class="">current generation volumes (gp2/io1/st1/sc1), and the live changes supported for </div><div class="">volumes attached to current generation instances, etc.</div><div class=""><br class=""></div><div class="">Finally, I think AWS is a powerful platform. More and more features will be </div><div class="">added for the sake of better performance of computing. There are too many </div><div class="">things need to learn in this field, so hopefully, I will keep learning and keep </div><div class="">updating.</div><div class=""><br class=""></div><div class="">Thanks for the reading!</div><div class=""><br class=""></div><div class="">-Tiankai Xie (txie4)</div><div class=""><br class=""></div><div class="">Links for “AWS Storage Day”:</div><div class=""><br class=""></div><div class="">Optimizing Storage for Big Data/Analytics Workloads: </div><div class=""><a href="https://www.slideshare.net/AmazonWebServices/optimizing-storage-for-big-data-analytics-workloads" class="">https://www.slideshare.net/AmazonWebServices/optimizing-storage-for-big-data-analytics-workloads</a></div><div class=""> </div><div class="">Migrating Large Scale Data Sets to the Cloud: </div><div class=""><a href="https://www.slideshare.net/AmazonWebServices/migrating-large-scale-data-sets-to-the-cloud" class="">https://www.slideshare.net/AmazonWebServices/migrating-large-scale-data-sets-to-the-cloud</a></div><div class=""> </div><div class="">Best Practices for Building a Data Lake on AWS: </div><div class=""><a href="https://www.slideshare.net/AmazonWebServices/best-practices-for-building-a-data-lake-on-aws" class="">https://www.slideshare.net/AmazonWebServices/best-practices-for-building-a-data-lake-on-aws</a></div><div class=""> </div><div class="">Supercharging the Value of Your Data with S3: </div><div class=""><a href="https://www.slideshare.net/AmazonWebServices/supercharging-the-value-of-your-data-with-amazon-s3" class="">https://www.slideshare.net/AmazonWebServices/supercharging-the-value-of-your-data-with-amazon-s3</a></div><div class=""> </div><div class="">Strategic Uses for Cost Efficient Long-Term Cloud Storage: </div><div class=""><a href="https://www.slideshare.net/AmazonWebServices/strategic-uses-for-cost-efficient-longterm-cloud-storage" class="">https://www.slideshare.net/AmazonWebServices/strategic-uses-for-cost-efficient-longterm-cloud-storage</a></div><div class=""><br class=""></div><div class="">Simple, Scalable and Highly Durable NAS in the Cloud – Amazon EFS: </div><div class=""><a href="https://www.slideshare.net/AmazonWebServices/simple-scalable-and-highly-durable-nas-in-the-cloud-amazon-efs" class="">https://www.slideshare.net/AmazonWebServices/simple-scalable-and-highly-durable-nas-in-the-cloud-amazon-efs</a></div><div class=""><br class=""></div></body></html>