18-649 Project 13

Check the course webpage for due dates

Please submit all project-related correspondence to

Changelog:

You have some additional time to complete this project. This is so that you can juggle all the other project demands being placed on you by other courses. We strongly advise you to set aside some time to do a thorough review of your design documents before handing in your final project.  REMEMBER: this project phase has a heavy weighting in the course grade!

Assignment:

There are no design changes required for this project.  You will simply complete the testing that you started last week, follow up on any run-time monitoring issues, and submit a final design portfolio.

1.  Complete Testing of Your Elevator Design

You must run all tests on your latest design:
 Note:  You must pass all acceptance tests in order to receive a grade in this course. There must also be zero failed assertions (after startup transients) from your Runtime Monitor. That means there can be no violations of the high level requirements.

1.1 Demo

In order to show that your elevator does indeed work, we ask you to perform a demo for the TAs. The demo procedure is as follows. You must pass all acceptance tests in order to pass the Demo. There must also be zero failed assertions (after startup transients) from The Course Staff Runtime Monitor. All these tests are randomly generated by the TA test generation script. In the event that your elevator does not pass the demo, the TA will provide you with the test and stats file (containing the seed to replicate the error).

During the medium length test, your TAs will discuss with you some aspect of your elevator. All group members must participate in this discussion in order to receive full credit for the demo. You will not need to stay for the long test once your discussion of the elevator has concluded. Your TA will contact you regarding the results of the long test.

The demo procedure check list, followed by the TA's can be found here.

Teams will be limited to TWO practice runs. Then there will be one final demo day that will be held the day before the project is due. After two practice runs, if the third run is unsuccessful, penalties will accrue for subsequent runs. There is also a late penalty that applies after the demo deadline on Monday the 8th. Your penalty for the project is the maximum of these two penalties and the penalty for a late portfolio handin. Any group member not present for the demo will continue to accrue late penalties until the demo. Demos that they did not attend still count towards their total attempts.

Most teams should be able to succeed with one or two practice runs. The point is that TAs are not there to debug your project. You should honestly believe you will pass before asking for a practice run. It is a good idea to generate a large number of randomized acceptance tests for your elevator before attempting even the first practice run.

Final demos are scheduled for the day before final project hand-in. All group members must be present for the demo. TAs will only grade one hand-in per team. If you tell them "grade this one" for an early demo, you forfeit any right to any further hand-in attempts. (These rules are in place to prevent abusive demos and hand-ins. If you have a special situation please let us know.)

2.  Submit a Complete and Consistent Design Portfolio

For this final project, you must submit a complete and consistent design portfolio.  This includes (but is not limited to)
The above list is only a partial list to jog your memory.  If you are in doubt about your design portfolio, you should review the guidelines provided by past project writeups.  You should also take a look at the detailed grading criteria for this project.

3.  Extra Credit

Note:  To be eligible for any extra credit, your elevator must be passing all unit, integration, and acceptance tests listed above and it must be submitted on time.  Projects that are submitted late or that are not passing all tests will not be considered for any bonus listed here.

3.1  Performance Competition

We will use the two performance metrics specified in the requirements document and output by the simulator in the .stats file (lower is better for performance; higher is better for satisfaction):
Delivery Performance=4 * (average passenger delivery time) + (maximum passenger delivery time)
Satisfaction Performance=100 - (various satisfaction metrics * x number of times violated); similar to delivery performance, the sum is 4 times average plus minimum.

An unspecified number of workloads (.pass files) will be used in evaluating total elevator performance. Some will be acceptance tests you have already seen, and some will be new. NOTE: If your elevator crashes during any test or fails to deliver all passengers in any test, you will be disqualified from the competition. Groups will be ranked (1=best, 10=worst) for each workload for each performance metric. Your overall group score will be the sum of your ranks for all workloads. Of the eligible groups, the one with the lowest sum will be the winner and all members thereof will be awarded 1 percentage point on their final course grade.
Ties will be broken based on best Satisfaction Performance score.

We will publish the performance scores on blackboard after all the projects have been submitted.  We will not publish any performance statistics from other groups ahead of time.

You can take a look at previous years' performance results here: Previous Performance Scores. Keep in mind, these do not necessarily reflect the performance that groups will have for your class.

3.3  Fault Tolerance

Implement fault tolerance in the case of dropped messages.

Modify your elevator so it delivers all passengers with dropped messages enabled and to work correctly with the new drive speed. By now you should know the design process you should use.

To earn extra credit your elevator must work correctly when 25-50% of the messages are dropped. No two messages of the same type are dropped in a row. For Example: A drop rate of 100% will drop every other packet

Your elevator must continue to meet all of the high-level requirements we have provided you both in the presence and absence of faults. Although your performance when faults occur is relative, you are encouraged to deliver passengers as quickly as possible.

To turn in your fault tolerant design, you should place a seperate elevatorcontrol folder in the fault_tolerance section of your portfolio. This copy of your code will only be used to check for fault tolerance, and not for the performance testing

  Injecting faults

You can inject your own message drop faults by using a .fault file like basicfault.fault 

In order to run your acceptance test with this fault file use the command:
java simulator.framework.Elevator -pf basicpass.pass -ff basicfault.fault
-ff allows definition of a fault file
adding the -dropv flag causes verbose output of dropped messages(useful for debugging)

3.2  Complete Design Package

By the end of the semester, you will have produced a hefty design package. Producing a complete design package of this size is a commendable feat, and we will reward those who have accomplished it. If we judge your portfolio to be complete and consistent, then all members of your team will receive the bonus

Note: only one group can win the performance competition, but every group can win the portfolio bonus! The message is: GOOD PROCESS IS OFTEN A MORE IMPORTANT GOAL THAN GOOD PERFORMANCE!

Handing In Results

Grading:

Here's the minimum requirement spread sheet. Though there are no constraints on how you assign work for this project, you must still fill out the minimum requirements sheet with your hours since project 12. The project will be late until this sheet is handed in. We advise that you divide the work as evenly as possible, but we are more concerned with you handing in a consistent design portfolio and working code.
This project counts as one team grade. A detailed grading rubric is available here. (PDF)
The rubric contains the detailed end-to-end traceability checks.  You should use this as a checklist for your final handin.  Remember that this project is worth considerably more than previous projects.

This project is worth 175 points + up to 2.5 points bonus on your final grade:

Note 1:  It is important that you submit a complete and consistent end-to-end design package.  In addition to the points awarded above for traceability and meeting portfolio requirements, TAs can deduct points from the project for failing to follow turn-in directions or having an end-to-end design package with blatant gaps.
Note 2:  Remember that a working final project is required to complete the course. It is especially important that you "make it work" for this final project!  This means that your elevator must be able to pass all acceptance tests (as outlined in the grading criteria above) before you will receive a grade in this course.


Back to course home page