Sample menu:

FAQs will be posted here. Keep checking this page.

Why don't we use piazza for 746 ?

Some classes use piazza. Based on our experiences, we choose to rely instead on a staff-wide mailing list, where we know that we are able to manage consistency and accuracy of answers that students get. We combine it with this FAQ page, which lets us curate the set of questions/answers that get promoted for attention by the entirity of the class. We have found that this combination allows us to more efficiently provide effective assistance to 746 students.

Do we get any grace days? How many?

You have a total of three grace days (i.e., unpenalized late days) for the entire semester.

How many submissions do we have per checkpoint? Autolab shows infinite submissions remaining.

You are allowed 25 submissions per checkpoint. Any additional submission after the first 25 submissions will incur a penalty of 10% of that checkpoint grade. Autolab shows infinite submissions because we do not impose a hard limit, but rather have a penalty per submission. You can easily check the number of submissions you have already made by scrolling down the checkpoint-submission page, as well as looking at your handin history via 'View Handin History' on the left panel.

LAB 2 (CloudFS) is out

Question 1: Are the slides on Project 2 available?

Answer:
Yes. See the lectures web.

 


Question 2: Is C++ okay?

Answer:
Yes. You can develop your project in C++. So long as you can compile, build and run the project successfully on the VM we have supplied (without installing any additional tools), your project will be graded on Autolab. You may have to change the Makefile to acheive this. You are not allowed to develop cloudfs in languages other than C / C++.

 

Question 3: How to debug my cloudfs outside the tests provided?

Answer:
1) initialize your cloudfs -> ./reset.sh and then ./mount_disks.sh at /src/scripts
2) run s3 server -> ./run_server at /src/s3-server/
3) Uncomment the line that adds the -f parameter to cloudfs in cloudfs_start (this parameter allows us to run cloudfs in foreground). Run make.
4) Run cloudfs in foreground -> ./cloudfs —no-dedup (for checkpoint 1)) at /src/build/bin/.
5) cd ~/mnt/fuse. Now run applications or system calls like touch, echo, cat, ls, chmod etc. Observe the system calls invoked, how your cloudfs responds.

 

Question 4: Using extended attributes gives me a ENOTSUP.

Answer:
You need to specify the attribute class to be able to correctly set an extended attribute.
Please consult the attr man page.

 

Question 5: lost+found directory causing tests to fail?

Answer:
lost+found is a directory created by the Ext4 formatting tools. In fact a lost+found exists when you format a filesystem as any of the Ext filesystems (it may even exist on your own machines!). In our scripts you are required to ignore lost+found, i.e. correctness requires you to *not* enlist lost+found as one of the existing folders when an 'ls' is performed on the FUSE mountpoint. The changes required to ignore lost+found must be made entirely in your FUSE implementation. Please do not modify the scripts you are provided as doing that might result in autograding problems on Autolab.

 

Question 6: I keep seeing 'Transport Endpoint not connected' errors. What's going on?

Answer:
This generally indicates that something in your code is broken, and has caused CloudFS to throw an error, return a bad status, or crash in some way or the other. (This generally translates to a segmentation fault because of which CloudFS has crashed). The FUSE module is unable to reach your user-level code and can't send it captured filesystem events. You need to log sufficient information (locally, please!) to catch which function is failing. Running a ./reset.sh in the scripts directory will get stuff back to normal.

 

Question 7: How am I supposed to log in cloudfs?

Answer:
Logging done in /tmp/cloudfs.log will be relayed to you via Autolab. Remember, logging too much can crash autolab and the results of too much logging are unpredictable.

 

Question 8: How to submit code for cloudFS ?

Answer:
You need to upload a .tar of the src folder.
tar -cvf src.tar src/

 

Question 9: How do I run the provided tests?

Answer:
Please run "make test_1_n" from the src/ directory, where n is the test number.

 

Question 10: My test cases fail with the following error: "Autodriver: Job timed out after 800 seconds". What is causing the timeout?

Answer:
This error most likely is because of a bug in your code. It could be because of a variety of reasons including an infinite loop in your code, an unimplemented function in your File system causing FUSE to not return to the OS, etc. To debug this issue, test all the Cloudfs functions locally before submitting to autolab.

 

Question 11: Unmounting fails in AutoLab?

Answer:
Seeing "Unmounting failed. Maybe it is not mounted?" is normal, and your tests still execute properly on AutoLab.

 

Question 12: My test cases fail with "Operation not permitted" error. What is causing this error?

Answer:
The most likely reason for "Operation not permitted" is that you are an incorrect error code. It can happen if one of your FUSE functions returns an error code (say, ENOPERM) that is different from the error code that is expected from a functional file system (see Question 13).

 

Question 13: How do I know the correct functionality expected out of my FUSE functions? (Or, how do I debug logical error?)

Answer:
CloudFS should appear exactly like any other file system (to the OS and users). One way to understand the expected functionality is to write small C programs that make system calls to the file system and test the program with (say) the ext4 filesystem directly and your CloudFS file system. The two file systems should appear to behave exactly the same (irrespective of what CloudFS does under the covers). This means that not only should CloudFS succeed on the system calls that should succeed, but if it fails, it should return the same error code that (say) the ext4 filesystem returns. A good way to identify the system calls you should be testing from your C program is to use the strace tool to identify the system calls being made by the test cases.

 

Question 10: Hash Header file

Answer:
In order to maintain hashes in cloudfs implemented in C, uthash.h can be used. If you are using C++ there are hash table functions in STL, called maps.

For usage refer to : http://troydhanson.github.io/uthash
Students are allowed to use only uthash.h from this link.

One student asked and was granted permission to use this file. Others can do the same.
Decision to use this file is entirely the responsibility of the student.

 

LAB-1

Question 1: Do we have sudo access on the AWS VM, to install / upgrade packages?

Answer: No, we have installed all the libraries that you would need to complete the project successfully. It would not be advisable to add any other libraries since the autolab instance to grade your submission won’t have it installed. If you still feel like you need any other library, let us know and we will look into it.

 

Question 2 : How do I submit my code to autolab?

Answer:
First of all, as the handout says you can only modify the myFTL.cpp file. All your changes go into this. When you want to submit the code on autolab, just submit this file in autolab.

 

Question 3:Why is there block-level mapping in the overprovisioned space, and not page-level mapping?

Answer:
The reason is that if we map pages of multiple data blocks into a single log-reservation block, when we have to clean, all the data blocks have to be erased (the ones whose pages are in the log-reservation block). Erases are at least an order of magnitude slower than reads / writes. This introduces sudden spikes of latency which are undesirable. There are ways to get around this spike, but I believe they are too complicated for this project. So, the policy we have chosen here is to choose a log-reservation block to buffer rewrites to a single data block, i.e. a log-reservation block cannot buffer writes from multiple data blocks.

 

Question 4 :Do the configuration knobs (ssd size, package size, die size, plane size and block size) have to be powers of 2?

Answer:
No, it is not necessary that they be powers of 2.

 

Question 5 :Will the overprovisioned percentage be specified such that it results in a partially overprovisioned block?

Answer:
No. For this project, the overprovisioning will always be block-aligned, i.e. there will never be a case where a block is partially overprovisioned, while partially being a data block.

 

Question 6 :Is your FTL supposed to support multi-threaded tests?

Answer:
No, for the purpose of this project, the tests will be single-threaded.

 

Question 7 :Can I access/modify the emulator classes?

Answer:
No. You are not supposed to access or modify the emulator classes in any way. The provided instance of ExecCallBack is sufficient to perform everything you may need to do in addition to the mapping.

 

Question 8 :Will my code be graded for style?

Answer:
Yes, your code will be graded for style. We will use the standard 15-213 coding standards for grading your code.

 

Question 9 :Can I use C++ standard library or other libraries?

Answer:
You are free to use C++ standard library. But if you have to use libraries other than the standard library, consult the course staff first.

 

Question 10 :I am seeing 0's for all the tests on Autolab, but the tests provided with the handout pass when I test locally. What is the problem?

Answer:
If for one of the tests in Autolab, your code hits an infinite loop (or something else) which causes the Autolab test to time out (each test has some fixed time out), you will get a 0 on all of the tests, even though some of the earlier tests might have passed. Please check your code for potential infinite loops. If you are sure that the problem is with Autolab, please contact the course staff.

 

Question 11 :How will the score of checkpoint 3 translate to partial credit, if below the target of 800?

Answer:
Suppose you have score s out of 1200 in scoreboard. You will get (s - 700)% of the credit in checkpoint 3, with minimum 0% and maximum 100%. Be warned that, if you score 0 points on some of the test cases because of fatal bugs, we will likely deduct points on the code quality component of the project 1 scoring.

Using Amazon Web Services (AWS)

Read the following guide before starting on any project development, and make sure your AWS account is working.

Setup and Billing


  1. Create an AWS account here if you don't have one already. Make sure you use your Andrew email address when you create your AWS account
  2. Once you have created your AWS account, please go ahead and join AWS Educate. To do that, click here and select "Join AWS Educate today", then "Apply for AWS Educate for Students". Join AWS Educate only if you have not done so previously for a different class Designate that you are a student, and fill the form that comes up.
    Note: Some of you may also be taking the 15-619 (Cloud Computing) class, and your Andrew email might already be tied to it. If that’s the case, use a different email address (e.g., @cs.cmu.edu, @cmu.edu, @gmail) to create a new AWS account.
  3. If your account is new, it comes with 750 hours/month (for 12 months) of free instance time. You need to use the "Free-Tier" instances in order to take advantage of that promotion.
  4. If you need AWS credits for the course, enter your Andrew ID in this Google spreadsheet. If you are registered with the class we will be emailing you a $50 voucher. You can start using AWS without the voucher, since Amazon will not bill you until the end of the month.
  5. To monitor your billing and usage, sign in to your AWS account, click on the drop-down menu with your account name and click on "Billing & Cost Management".
Warning: Make sure to protect your AWS credentials! Do not share or expose them to others online. If your AWS account is compromised and unauthorized charges are made to your credit card, we will be unable to help.

Starting your AWS instance


  1. Sign into your AWS account.
  2. Click on EC2 under "Compute Services" to be taken to the EC2 Dashboard.
  3. On the right corner of the top bar, click the second drop-down menu from the right to select the datacenter you will be using. Select "N. Virginia".
  4. In the sidebar on the left, click on "AMIs" under the "Images" group.
  5. Click on the drop-down menu under "Launch" and select "Public images".
  6. Use the bar to the right of the drop-down menu to search for '746-vm'. Double-check that the image owner is '169965024155'. This is the VM image for the course. Use the refresh button at the top right if the image doesn't show up right away.
  7. Select the AMI, and click "Launch".
  8. Make sure the "t2.micro" instance type is selected (which is also Free-tier eligible, if you are using a new account that benefits from that promotion).
  9. Click "Review and Launch".
  10. On the next screen click "Launch".
  11. In the first drop-down menu select "Proceed without a key pair", and check the checkbox. To access the instance, you will need this key.
  12. Click "Launch Instances", and on the next screen click "View Instances"
  13. In the EC2 Dashboard Instances screen wait for your VM instance state to transition to "running", and the VM status checks to indicate "2/2 checks passed".
  14. Select your instance and check the 'Public DNS' column. Make a note of the machine's FQDN, which will be of the form X.compute-Y.amazonaws.com
  15. From your terminal, run:

    ssh -Y -i "path/to/746-student.pem" student@X.compute-Y.amazonaws.com

    • Make sure the path passed to the -i option points to the 746-student.pem key pair you downloaded from the course website
    • Remember to run "chmod 400 746-student.pem" to ensure your key pair file carries the right permissions
  16. Run the ssh command to connect to your instance!

Securing your AWS instance


It is a good idea to change the key you use to login to your AWS instance, and stop using the 746-student.pem which the AWS template comes configured with. To do so, we provide a simple script: make_secret_key.sh which will generate a new key and use it to replace 746-student.pem on your instance. Note that if you terminate your instance and create it again from the template image, you will have to run this script again.


Terminating your AWS instance


When you are done using your VM, you can stop it or terminate it. You can stop your VM by running the shutdown command from within Ubuntu, or using Actions → Instance State → Stop.

Be warned that when your VM is stopped it still consumes resources, which will lead to usage charges for EBS storage (and you may run out of your credit faster). Instead of stopping your VM, make sure to copy out your source code when you're done, and then terminate your instance. You can do that via Actions → Instance State → Terminate.