The AAA Foundation for Traffic Safety tasked the MIT AgeLab with developing a data-driven system for rating the effectiveness of new in-vehicle technologies intended to improve driver safety. Such a system was envisioned as having the potential to educate and guide consumers towards more confident and strategic purchasing decisions, ideally encouraging adoption of technologies showing demonstrated safety benefit. Further, an evaluation of the status and extent of existing data was seen as a way of identifying research gaps in the present state of knowledge about these safety systems. It should be made clear that the focus of the project based on the mandate given to the MIT AgeLab was on given technologies as a class, not on a rating review of individual vehicle model implementations. Development of the rating system and identification of data was undertaken in consultation with identified academic, industrial, consumer, NGO, and governmental experts as well as with representatives of a majority of the major automotive manufacturers. Almost universal endorsement of the importance of this undertaking was voiced.
A top level rating structure has been developed that independently considers safety benefit potential and objectively demonstrated benefit. The latter values are often found to be lower than theoretical expectations. Factors that may be relevant to understanding why such differences appear have been identified as part of the overall project. The emphasis on ratings based on observed benefit for actual drivers under real-world conditions is a key aspect of why this system complements, rather than competes with, ratings developed by IIHS and NCAP which focus largely on controlled test track evaluations of engineered capability. In addition, the rating structure assesses benefit relative to overall crash, injury, and fatality rates – and in relation to the specific scenario / crash event type that a given technology is intended to address. This allows consumers to consider a technology relevant to their particular driving needs.
A total of seven technologies have been reviewed – two reference technologies (Electronic Stability Control and Adaptive Cruise Control) and five emergent safety technologies (Adaptive Headlights, Back-Up Cameras, Forward Collision Warning, Forward Collision Mitigation, and Lane Departure Warning). A major finding of the project has been that only relatively limited data is available upon which to objectively rate the real-world performance of most of these safety systems. A number of experts and industry representatives expressed some surprise at both the divergence between theoretical and observed benefit and the relative scarcity of data upon which to make objective assessments, while others were quite aware of these issues and the need for the development of objective data on real-world performance. This undertaking appears to have already succeeded in one of its goals by stimulating substantive constructive discussion and engagement within the research and industry based safety communities.
The report begins with a review of the original project objectives and an overview of key activities undertaken during the course of the project. Selected observations on the development of the proposed rating system follow, including a brief discussion of the evolving view of rating factors and concepts of scaling that were considered as the project developed. The issue of rating a technology class in contrast to rating specific vehicle implementations is summarized. Key concepts in the proposed rating system are then presented. In particular, the concepts of projected vs. demonstrated benefit and overall safety benefit vs. scenario specific benefit are discussed. A proposed approach to scaling ratings is presented. Detailed reviews of available data on each of the technologies were developed as a core component of the project and used to identify values to rate the technologies. The benefit values extracted from the reviews to assign ratings to each technology class are described. To appreciate the full context from which the ratings are drawn, readers are encouraged to see the more extensive technology review summaries provided in Appendix C. These reviews not only consider the issue of evaluating technology effectiveness, but also address topics including consumer awareness and trust, mobility significance, technology penetration, frequency of use, training and educational issues associated with the technologies, behavioral adaptation, demands placed upon the driver, vehicle type considerations, limitations and failure conditions, and differences between technology implementations. Several approaches to summarizing the ratings in a matrix format are presented. These are intended to be conceptual in nature rather than necessarily representing exact design format recommendations.
The report then presents a number of observations arising out of the work on the project. As already noted, a major point of discussion is the issue of the relatively limited body of objective data that is available to advise not only the consumer, but also the wider public and automotive industry on how these emerging safety technologies are actually performing. Also emphasized is the point that what is presented in this report is a proposed rating system; it is intended to encourage discussion and consideration of important issues related to the better understanding of safety technologies. The extent to which the approach and values proposed here are further developed and updated to keep pace with rapidly evolving technologies and increased scientific evaluation of their performance is seen as an open question. An extended consideration of limitations and points to be kept in mind follows. Comments and critiques from industry reviewers and various Advisory Panel members are recognized and integrated in this important section