For some class meetings, readings will be assigned. Usually, these
readings will consist of relevant technical papers, articles or
instructor-prepared notes. Paper copies of assigned readings and
notes will be provided in class and online. However, please note that
online versions of the readings are only available when accessed from
a 128.2.* (CMU) IP address or a local-only CMU IP address.
The readings listed should be read BEFORE class on the assigned
day.
April 16th (L19): Storage Virtualization
- Recommended optional reading:
Serverless Network File Systems (xFS). Thomas
E. Anderson, Michael D. Dahlin, Jeanna M. Neefe, David A. Patterson,
Drew S. Roselli, Randolph Y. Wang. Appears in the ACM Transactions on
Computer Systems, Vol. 14, No. 1. February 1996. (pdf)
April 11th (L18): Multi-core issues for FS/storage
- An Analysis of Linux Scalability to Many Cores. Silas Boyd-Wickizer, Austin T. Clements, Yandong Mao, Aleksey Pesterev, M. Frans Kaashoek, Robert Morris, and Nickolai Zeldovich. Appears in OSDI 2010.
(PDF)
April 9th (L17): More reliability techniques
- Architectures and Algorithms for On-Line Failure Recovery in Redundant Disk Arrays. Mark Holland, Garth A. Gibson, and Daniel P. Siewiorek. Appears in the Journal of Distributed and Parallel Databases, Vol. 2, No. 3, July 1994.
(available here)
- Scalable performance of the Panasas Parallel File System. (from Lecture 15 below)
April 4th (L16): Backup and disaster recovery
- Designing for disasters. Kimberly Keeton, Cipriano Santos, Dirk Beyer, Jeffrey Chase, John Wilkes. Appears in the Proceesings of the Third Usenix Conference of File and Storage Technologies (FAST'04).
(pdf)
April 4th: Guest lecture by John Bent (reading optional, since tornados forced cancel)
- PLFS: A Checkpoint Filesystem for Parallel Applications. John Bent, Garth Gibson, Gary Grider, Ben McClelland, Paul Nowoczynski, James Nunez, Milo Polte, Meghan Wingate. Appears in Supercomputing, November 2009. (link to paper)
April 2nd: Parallel File Systems
- GPFS: A Shared-Disk File System for Large Computing Clusters. Frank Schmuck and Roger Haskin. Appears in FAST, January 2002. (pdf)
- Scalable performance of the Panasas Parallel File System. Brent Welch, Marc Unangst, et al. Appears in FAST, February 2008. (pdf)
March 26th (L14): Storage for data-intensive computing
- The Google File System. Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung. Appears in ACM Symposium on Operating Systems Principles (SOSP), 2003.
(pdf)
- Bigtable: A Distributed Storage System for Structured Data. Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber. Appears in USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2006.
(pdf)
- Recommended optional reading:
- MapReduce: Simplified Data Processing on Large Clusters. Jeffrey Dean and Sanjay Ghemawat. Appears in USENIX Symposium on Operating Systems Design and Implementation (OSDI), 2004.
(pdf)
March 21st (L13): NAS interfaces (a.k.a. DFS II)
March 19th (L12): Distributed file systems
- The Design and Implementation of the 4.4BSD Operating System (Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, and John S. Quarterman, 1996)
- Chapter 9 (The Network Filesystem) (pdf)
- Recommended optional reading:
- Scale and Performance in a Distributed File System. John H. Howard, Michael L. Kazar, Sherri G. Menees, David A. Nichols, M. Satyanarayanan, Robert N. Sidebotham, Michael J. West. Appears in the ACM Transactions on Computer Systems, Vol. 6, No. 1, Pages 51-81. February 1988. (pdf)
- Optional reading:
- Optional reading: "NFS Version 3 Protocol Specification" (RFC 1813:
B. Callaghan, B. Pawlowski, P. Staubach, June 1995) (txt)
March 7th (L11): Disk request optimization and storage-over-IP
- Scheduling Algorithms for Modern Disk Drives. Bruce L. Worthington, Gregory R. Ganger, and Yale N. Patt. Appears in the Proceedings of the ACM Sigmetrics Conference. May, 1994. (ps)
- For more on incast: http://www.pdl.cmu.edu/Incast/
February 27th and 29th (L10): Flash Storage
February 22nd (L9): Disk array systems
- RAID: High-Performance, Reliable Secondary Storage (Peter M. Chen,
Edward K. Lee, Garth A. Gibson, Randy H. Katz, and David
A. Patterson, 1994) (same as L8)
February 20th (L8): Disk array organization
- RAID: High-Performance, Reliable Secondary Storage (Peter M. Chen,
Edward K. Lee, Garth A. Gibson, Randy H. Katz, and David
A. Patterson, 1994)(pdf)
February 13th (L7): Caching and FS integrity
- Soft Updates: A Solution to the Metadata Update Problem. Gregory R. Ganger, Markall Kirk McKusick, Craig A.N. Soules, Yale N. Patt. ACM Transactions on Computer Systems. May 2000. (pdf)
- Practical File System Design with the Be File System (Dominic Giampaolo, 1999)
- Chapter 7 (Journaling) (pdf)
- Verifying File System Consistency at Runtime. Daniel Fryer, Kuei Sun, Rahat Mahmood, TingHao Cheng, Shaun Benjamin, Ashvin Goel, Angela Demke Brown. Conference on File and Storage Technologies (FAST), 2012. (pdf)
February 8th (L6): On-disk data layout
- The Design and Implementation of the 4.4BSD Operating System (Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, and John S. Quarterman, 1996)
- Chapter 8 (Local filestores) (pdf)
- Embedded Inodes and Explicit Grouping: Exploiting Disk Bandwidth for Small Files. Gregory R. Ganger and M. Frans Kaashoek. Appears in the Proceedings of the Usenix Technical Conference. January, 1997. (pdf) (ps)
February 6th (L5): File system organization
- Practical File System Design with the Be File System. Dominic Giampaolo. 1999.
- Chapter 2 (What is a file system?) (pdf)
- UNIX Internals: The New Frontiers. Uresh Vahalia. 1996.
- Chapter 8 (File system interface and framework) (pdf)
February 1st (L4): Disk drive firmware
- An Introduction to Disk Drive Modeling. Chris Ruemmler and John Wilkes, 1994. (pdf)
- Microbenchmark-based Extraction of Local and Global Disk Characteristics. Nisha Talagala, Remzi H. Arpaci-Dusseau, and David Patterson (pdf)
January 25th and 30th (L3): Disk drive operation
- Digital Large System Mass Storage Handbook. Paul Massiglia. 1986.
- Probability Refresher. Mor Harchol-Balter. 2000. (pdf)
January 23rd (L2): Metrics of I/O system quality
- Computer Architecture: A Quantitative Approach, 3rd ed. John L. Hennessy and David A. Patterson. 2002. (pdf)
- Section 7.7: "I/O performance measures"
- Section 7.8: "A Little queuing theory"
- Section 7.9: "Benchmarks of storage performance and availability"
- MTBF Description, Kevin Dally. 1995. (txt)
January 18th (L1): Introduction and overview of storage systems
This first meeting will be more than just organizational in nature.
We will discuss how the class is going to work and what will (and won't)
be covered.
See the 18-746 overview for a recap of the
general information.
We will also dive into the course by overviewing the area of storage
systems.