UL 4600 Guidebook URL list

Link to this page: https://users.ece.cmu.edu/~koopman/ul4600/UL4600_Guidebook.html
If a link is dead, try it through the Wayback Machine here:https://archive.org/

The UL 4600 Guidebook: What to Include in an Autonomous Vehicle Safety Case, 2022.
ISBN: 9798365303065 Trade Paperback
ISBN: 9798365303249 Hardcover
ISBN: 9781005675004 E-book for non-Amazon distributors
ASIN: B0BNLVC22J Kindle ebook

(See bottom of page for errata)

Chapter 1:
- UL 4600 information launch page:
  https://users.ece.cmu.edu/~koopman/ul4600/index.html
- Video tutorial on UL 4600 (23 minutes):
  - YouTube version: https://youtu.be/ZxVMX8SjPvw
  - Archive.org version: https://archive.org/details/L109-ul-4600
- Video tutorial series on AV safety to provide background
  - https://users.ece.cmu.edu/~koopman/lectures/index.html#av
    (includes slides, YouTube videos, and archive.org mirrors)
- A graduate-level course on embedded system and software safety taught by the author at Carnegie Mellon University, with all lecture videos freely available online:
  https://course.ece.cmu.edu/~ece642/
- Some historical notes on the evolution of UL 4600:
  https://www.eetimes.com/safe-autonomy-ul-4600-and-how-it-grew/
Chapter 2:
- ISO 21448: https://www.iso.org/standard/77490.html
- ISO 26262: https://en.wikipedia.org/wiki/ISO_26262
- ISO/PAS 8800: https://www.iso.org/standard/83303.html
- MIL-STD-882: https://www.sebokwiki.org/wiki/System_Safety
- SAE J3016: https://users.ece.cmu.edu/~koopman/j3016/
- SAE J3018: https://www.sae.org/standards/content/j3018_202012/
- TR 4804 (pre-standard SaFAD report version): https://www.aptiv.com/docs/default-source/white-papers/safety-first-for-automated-driving-aptiv-white-paper.pdf
- UL 4600 does not provide an answer to the question of how safe might be "safe enough." That question is discussed in Koopman, "How Safe Is Safe Enough," 2022: https://safeautonomy.blogspot.com/2022/09/book-how-safe-is-safe-enough-measuring.html
Chapter 5:
- ACWG, Assurance Case Guidance, Challenges, Common Issues and Good Practice, SCSC-159, Ver. 1, Aug. 2021. https://scsc.uk/r159:1
- ACWG, Goal Structuring Notation Community Standard. https://scsc.uk/gsn?page=gsn%202standard
- Goodenough, Weinstock & Klein, Eliminative Argumentation: A Basis for Arguing Confidence in System Properties, CMU/SEI-2015-TR-005, 2015. https://www.researchgate.net/publication/272678149_Eliminative_Argumentation_A_Basis_for_Arguing_Confidence_in_System_Properties
- Koopman, P., Kane, A. & Black, J., "Credible Autonomy Safety Arguments," Safety-Critical Systems Symposium, Bristol UK, Feb. 2019. https://users.ece.cmu.edu/~koopman/pubs/Koopman19_SSS_CredibleSafetyArgumentation.pdf
- Stanford Encyclopedia of Philosophy, Defeasible reasoning. https://plato.stanford.edu/entries/reasoning-defeasible
- Wikipedia, Falsifiability. https://en.wikipedia.org/wiki/Falsifiability
- William S. Greenwell, John C. Knight, C. Michael Holloway, and Jacob J. Pease, "A taxonomy of fallacies in system safety arguments," Proc. Int'l System Safety Conference (ISSC), Albuquerque, NM, 2006. https://ntrs.nasa.gov/citations/20060027794
Chapter 6:
- ISO 26262:2018 part 3, Concept Phase
- Koopman, P., How Safe Is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety, September 2022.
  - Chapters 3, 4, 5, and 10 are relevant to managing risk and the important differences between risk and safety.
  - Chapter 9 proposes a list of topics that should be included in "safe enough" criteria.
  - See: https://safeautonomy.blogspot.com/2022/09/book-how-safe-is-safe-enough-measuring.html
- Lala & Harper, Architectural principles for safety-critical real-time applications, Proc. IEEE vol. 82 no. 1, Jan 1994, pp. 25-40. https://ieeexplore.ieee.org/document/259424
- Wikipedia, Automotive safety integrity level https://en.wikipedia.org/wiki/Automotive_Safety_Integrity_Level
- Wikipedia, Risk matrix, https://en.wikipedia.org/wiki/Risk_matrix
- Wikipedia, Safety integrity level, https://en.wikipedia.org/wiki/Safety_integrity_level
Chapter 7:
- Brooks, The Big Problem with Self-Driving Cars Is People, IEEE Spectrum, July 2017. https://spectrum.ieee.org/the-big-problem-with-selfdriving-cars-is-people
- IEEE Ethics in action/autonomous and intelligent systems. https://ethicsinaction.ieee.org/p7000/
- UK Law Commission, Automated Vehicles Final Report, 2022. https://www.lawcom.gov.uk/document/automated-vehicles-final-report/
Chapter 8:
- Data Safety Initiative Working Group (DSIWG), Data Safety Guidance v.3.4, SCSC-127G, Feb. 2019. https://scsc.uk/scsc-127G
- ISO 21448:2022, Road vehicles - Safety of the intended function.
- Koopman, Short course on AV safety. https://users.ece.cmu.edu/~koopman/lectures/index.html#av
- Koopman, P. & Fratrik, F., "How many operational design domains, objects, and events?" SafeAI 2019, AAAI, Jan 27, 2019. https://users.ece.cmu.edu/~koopman/pubs/Koopman19_SAFE_AI_ODD_OEDR.pdf
- Safety First for Automated Driving, 2019. https://group.mercedes-benz.com/dokumente/innovation/sonstiges/safety-first-for-automated-driving.pdf
Chapter 9:
- ANSI EIA-649-C 2019 Configuration Management Standard " Boehm et al., Balancing Agility and Discipline: A Guide for the Perplexed, 1st Edition, 2003
- Koopman, Better Embedded System Software, 2022 (revised edition). ISBN-13: 979-8596008050
- Koopman, 18-642 Course Lectures, Carnegie Mellon University. https://users.ece.cmu.edu/~koopman/lectures/index.html#642
- SEBOK: https://www.sebokwiki.org
- SWEBOK: https://www.computer.org/web/swebok
- SEI, "+SAFE, V1.2: A Safety Extension to CMMI-DEV, V1.2," https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=8219
- Wikipedia, Configuration management. https://en.wikipedia.org/wiki/Configuration_management
Chapter 10:
- Avizienis, A; Laprie, J.-C.; Randell, B.; Landwehr, C. "Basic concepts and taxonomy of dependable and secure computing," IEEE Transactions on Dependable and Secure Computing, Jan.-March 2004, Volume: 1 Issue:1 pp.: 11-33. https://doi.org/10.1109/TDSC.2004.2
- ISO/SAE 21434, Road Vehicles - Cybersecurity Engineering, 2021.
- Knight, John., Fundamentals of Dependable Computing for Software Engineers, 2017, ISBN 1138402222.
- Koopman, Safety Architecture Patterns: https://youtu.be/QEHr8J-ByLQ
- Wendover Productions, Small Planes Over Big Oceans (ETOPS Explained): https://youtu.be/HSxSgbNQi-g
Chapter 11:
- Driscoll, K., Hall, B., Koopman, P., Ray, J., DeWalt, M., Data Network Evaluation Criteria Handbook, AR-09/24, FAA, 2009. https://users.ece.cmu.edu/~koopman/pubs/faa09-24_data_network_evaluation_criteria_handbook.pdf
- SCSC Group: Data Safety Initiative: https://data-safety.tech/
  - DSI Data Safety Guidance, with the latest version being SCSC-127G version 3.4: https://scsc.uk/scsc-127G
Chapter 12:
- Kane et al., A case study in runtime monitoring of an autonomous research vehicle system, 2015. https://users.ece.cmu.edu/~koopman/pubs/kane15_monitoring.pdf
- Pegasus project: https://en.wikipedia.org/wiki/Pegasus_Project_(investigation)
- SEBok:
  - Verification: https://www.sebokwiki.org/wiki/System_Verification
  - Validation: https://www.sebokwiki.org/wiki/System_Validation
Chapter 13:
- Wikipedia, Commercial off-the-shelf: https://en.wikipedia.org/wiki/Commercial_off-the-shelf
Chapter 14:
- DOT/FAA/TC-15/33 Obsolescence and Life Cycle Management for Avionics
- ISO 28000:2022 Security and resilience - Security management systems - Requirements
- SEBoK: Logistics: https://www.sebokwiki.org/wiki/Logistics
Chapter 15:
- " SEBoK: System Reliability, Availability, and Maintainability: https://www.sebokwiki.org/wiki/System_Reliability,_Availability,_and_Maintainability
- Wikipedia, Maintenance: https://en.wikipedia.org/wiki/Maintenance_(technical)
Chapter 16:
- Koopman, P., How Safe Is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety, September 2022.
  - Chapter 8 covers a way to link SPIs to safety case claims in detail.
  - https://safeautonomy.blogspot.com/2022/09/book-how-safe-is-safe-enough-measuring.html
- Koopman, Safety Performance Indicators and Continuous Improvement Feedback (SEAMS Keynote talk), 2022
Chapter 18:
- Further information about UL 4600 can be found on this web page: https://users.ece.cmu.edu/~koopman/ul4600/index.html At the time of publication of this book, key resources presented on that web page include:
  - A downloadable voting draft copy of version 1 of UL 4600. Important changes have been made since that time. However, the general structure and substantial amounts of content remain similar or identical to the issued standard. This is a convenient way for educators to give students an easy-to-use copy of the standard, as well as for new users of the standard to get a convenient orientation free of charge. https://users.ece.cmu.edu/~koopman/ul4600/191213_UL4600_VotingVersion.pdf
  - Access to the free digital view of the current official version of the standard via the UL standards web site. There is a link to purchase an official copy, but the "digital view" link permits web viewing of the entire standard with no purchase. Free account registration is required. https://www.shopulstandards.com/ProductDetail.aspx?productid=UL4600
  - A video overview of the standard (20 minutes):
    - YouTube: https://youtu.be/ymGovH_K56U
    - Archive.org: https://archive.org/details/L109-ul-4600
    - A list of current voting STP members: https://csds.ul.com/Include/ViewRoster.aspx?GroupID=1862&stpNumber=4600
    - A frequently asked questions (FAQ) page: https://safeautonomy.blogspot.com/p/ul-4600-faq.html
- More general support material on safety critical system design lessons learned can be found here:
  - Compendium of automotive software recalls: https://betterembsw.blogspot.com/p/potentially-deadly-automotive-software.html
  - Computer-based system safety essential reading list: https://safeautonomy.blogspot.com/p/safe-autonomy.html
  - Koopman, Safe Autonomy Blog: https://safeautonomy.blogspot.com/
  - NASA Safety, Quality, Reliability, Maintainability library: https://standards.nasa.gov/safety-quality-reliability-maintainability
  - The RISKS Digest: https://catless.ncl.ac.uk/Risks/

Changes:

Ver. 1-00-02: on Dec. 14, 2022:
- Changed pg. 6, section 2.2.2:
  - From: "The stated scope of UL 4600 is any road vehicle that does have continuous human supervision for safety."
  - To: "The stated scope of UL 4600 is any road vehicle that does not have continuous human supervision for safety."
Ver. 1-00-03: on Dec. 19, 2022:
- Ebook formatting changes for Smashwords compatibility; does not affect other formats.

Last updated: 14 December 2022