Analyzing Safety Ratings and SafeStat Scores
by Darren Brewer
Carrier411 Services, Inc.
How many unsafe trucking companies do you still use? Most people who tender freight to motor carriers are unable to answer this simple but challenging question with 100% confidence. You must have the right answer though, if you want to protect your company, and there's only one answer. But how can you be sure?
Where to Begin the Process
There are protective measures that every broker, 3PL provider and shipper must employ, and knowing how to analyze the safety fitness of your carriers is the place to start. Hiring carriers with due diligence is a responsibility that requires your company to have a corporate policy for qualifying carriers, and that policy absolutely must include consideration for all available carrier safety data.
The consequences of hiring an unfit carrier can be devastating if the carrier you select is involved in a serious accident. In order to minimize risk, it is important that your employees understand and adhere to corporate policy when qualifying potential carriers and re-evaluating existing carriers for safety compliance - a monitoring process that should be done on a continual basis.
National 2005 Large Truck Crash Facts
- • 4,932 Large Trucks Involved in Fatal Crashes
- • 5,212 Fatalities in Crashes Involving Large Trucks
- • 139,772 Large Trucks Involved in Non-Fatal Crashes
- • 60,617 Large Trucks Involved in Injury Crashes
- • 91,824 Injuries in Crashes Involving Large Trucks
- • 79,155 Large Trucks Involved in Towaway Crashes
- • 2,371 Large Trucks Involved in Hazmat (HM) Placard Crashes
Where Do You Draw the Line?
Carrier411 conducted a recent survey involving some of the transportation industry's leading brokerages. Representatives were asked about the steps they take to qualify carriers based on safety considerations. Most all brokerages recognize the importance of selecting safe carriers and have recently modified their due diligence policies in an effort to reduce risk. Some created formal corporate policy for the first time. However, even in light of recent court decisions, many representatives expressed uncertainty about the best steps to take, stating there are "no hard and fast rules" established for the industry to follow.
Companies Surveyed Fell into Four Distinct Camps
1. Most brokers only evaluate carrier safety ratings to make their determination.
2. Others only consider SEA scores for driver safety and vehicle safety.
3. Fewer brokers look at the combination of safety ratings and SEA scores.
4. One company representative said evaluating carrier safety is a "waste of time" in a time-sensitive market, because their brokerage would be named in a lawsuit regardless of the carrier selection process.
So what is the best approach to qualify carriers with due diligence and maintain a qualified carrier base?
Safety Data Should Be Viewed Holistically
Because of the Schramm case involving C.H. Robinson, and other cases like it currently being litigated, safety ratings and SafeStat SEA scores are especially critical. As a matter of due diligence, you should consider all available carrier safety data (a combined analysis of safety ratings and SEA scores) in order to reduce or eliminate negligent hiring of unfit motor carriers.
The Federal Motor Carrier Safety Administration (FMCSA) provides carrier safety data through its Safety and Fitness Electronic Records (SAFER) System website. SAFER System access is provided free of charge to the Company Snapshot. It provides a concise electronic record of a carrier's identification, size, commodity information and safety record, including the carrier safety rating, roadside out-of-service inspection summary and crash information. Current carrier safety data is also incorporated into the Carrier411 website, along with helpful links to SafeStat SEA scores and other vital carrier information.
How Carrier Safety Ratings Are Determined
In simple terms, the FMCSA measures a carrier's compliance with the Federal Motor Carrier Safety Regulations (FMCSR) and the carrier is assigned a safety rating. Although this is a simple explanation, the methods it uses to collect and calculate safety data are elaborate.
On its website, FMCSA explains that motor carriers rated before 1986 were assigned safety ratings based on previous procedures used to determine the carrier's compliance with the FMCSR, such as a safety management audit or a re-investigation conducted on the carrier's operation.
Beginning in October 1986, carriers were assigned a safety rating based on a safety review or a compliance review. A compliance review is an overview of a motor carrier's knowledge of the FMCSR, which covers all major areas of the safety regulations.
A compliance review is a statistical sampling or audit of required records to determine a motor carrier's compliance with the FMCSR. And according to former FMCSA Administrator Annette Sandberg, the compliance review process is the core of the agency's enforcement program, consuming over 40 percent of its staff time. During a compliance review, performance factors (such as accidents, drivers and vehicles placed out-of-service and other violations) are examined and federal regulatory violations may be discovered.
One of Three Carrier Safety Ratings Can Be Issued
1. Satisfactory - Records indicate no evidence of substantial non-compliance with safety requirements.
2. Conditional - Records indicate that the carrier was out of compliance with one or more safety requirements.
3. Unsatisfactory - Records indicate evidence of substantial noncompliance with safety requirements.
Understanding Part § 385 and the Notification of Safety Fitness
(A) The FMCSA will provide a motor carrier written notice of any safety rating resulting from a compliance review as soon as practicable, but not later than 30 days after the review. The notice will take the form of a letter issued from the FMCSA's headquarters office and will include a list of compliance deficiencies, which the carrier must correct.
(B) If the safety rating is "satisfactory" or improves a previous "unsatisfactory" safety rating, it is final and becomes effective on the date of the notice.
(C) In all other cases, a notice of a "proposed safety rating" will be issued. It becomes the final safety rating after the following time periods: (1) For motor carriers transporting hazardous materials in quantities requiring placarding or transporting passengers by commercial motor vehicle - 45 days after the date of the notice. (2) For all other motor carriers operating commercial motor vehicles - 60 days after the date of the notice.
(D) A proposed safety rating of "unsatisfactory" is a notice to the motor carrier that the FMCSA has made a preliminary determination that the motor carrier is "unfit" to continue operating in interstate commerce, and prohibitions in § 385.13 will be imposed after 45 or 60 days if necessary safety improvements are not made.
(E) A motor carrier may request the FMCSA perform an administrative review of a proposed or final safety rating. The process and time limits are described in § 385.15.
(F) A motor carrier may request a change to a proposed or final safety rating based upon its corrective actions. The process and time limits are described in § 385.17.
If a carrier whose request for rating change is denied, it may request an administrative review by the FMCSA. The motor carrier must make the request within 90 days of the denial of the rating change request. However, if a proposed rating has become final, it remains in effect while the administrative review is in process.
SafeStat Data and Compliance Reviews
As explained on the SafeStat website, the Motor Carrier Safety Status Measurement System (MCMIS) is an automated, data-driven analysis system designed by FMCSA. Also known as SafeStat, it is essentially the FMCSA central database of all motor carriers with a U.S. DOT number.
Motor Carrier Safety Status (SafeStat) Measurement System data enables the FMCSA to quantify and monitor the safety status of individual motor carriers on a monthly basis, thereby enabling the agency to focus enforcement resources on carriers identified to pose the greatest potential safety risk. As a result, carriers are identified and prioritized for on-site compliance reviews. During a compliance review, SafeStat information is verified, such as number of power units and vehicle miles traveled, and the carrier's safety status is confirmed.
SafeStat combines current and historical carrier-based safety performance information to measure the relative (peer-to-peer) safety fitness of interstate commercial motor carriers and intrastate commercial motor carriers that transport hazardous materials. This information includes federal and state data on crashes, roadside inspections, on-site compliance review results and enforcement history.
Individual states are required to provide standard, basic information about large truck and bus crashes to the FMCSA within 90 days of the crash event, and the results of roadside inspections within 21 days of inspection. However, the completeness, timeliness and accuracy of crash data - and to a lesser extent roadside inspection data - vary from state-to-state. Accordingly, SafeStat's ability to assess the safety fitness of individual motor carriers accurately and objectively may be inconsistent and not conclusive without additional analysis.
Safety Evaluation Area (SEA) Scores
SafeStat uses safety data to measure the relative safety status of carriers in four Safety Evaluation Areas.
1. Accident (ACSEA) - Not Publicly Available
2. Driver (DRSEA)
3. Vehicle (VHSEA)
4. Safety Management (SMSEA) - Not Often Scored
SEA scores are only calculated for carriers with sufficient safety data related to each evaluation area, and the scores are a percentile rank (based on a scale of 0-100), with 0 as the best possible score and 100 as the absolute worst. The higher the SEA score, the worse the safety status. Each SEA is based on two or more indicators supported by different data sources.
SEA scores for a carrier are displayed as a bar chart on the SafeStat website, and the SEA values approximate the carrier's percentile rank relative to all other carriers with sufficient data to be assessed within that evaluation area. Deficient SEAs are greater than or equal to 75. As an example, a carrier with a Driver SEA score of 88 indicates that approximately 88% of the carriers with sufficient data had better driver performance than that carrier with respect to driver-related safety and compliance. The remaining 12% of carriers with a Driver SEA had a worse safety performance.
Realities of Carrier Safety Statistics
Consider the crash rates of carriers with and without high SEA values, as reported in the 2004 SafeStat Effectiveness Study Update.
|Safety Evaluation Area
||Number of Carriers
||Weighted Crash Rate*
|High Accident SEA
|No High Accident SEA
|High Driver SEA
|No High Driver SEA
|High Vehicle SEA
|No High Vehicle SEA
|High SM SEA
|No High SM SEA
* Number of weighted crashes per 1,000 power units from 3/2001 to 9/2002.
* Percent greater than carriers without high SEA.
Here are the published findings of the study:
Accident (ACSEA) - The results confirm what may seem intuitively obvious: carriers with high crash rates in the past are likely to continue to have high crash rates in the future. In other words, past crash rate performance is a good indicator of future crash rate performance. The effectiveness study shows a 169% greater post-selection crash rate for carriers with poor Accident SEAs compared to carriers that were not identified as having poor Accident SEAs. Comparing SEAs, the Accident SEA is by far the most effective SEA for identifying high-risk carriers. But unfortunately, it is not currently available for public viewing.
Driver (DRSEA) - The Driver SEA (with a 63% higher crash rate for carriers with poor Driver SEAs) is the next most effective SEA. These results from the study are especially impressive, because the criteria for the Driver SEA are based on violations and are independent of crash history.
Vehicle (VHSEA) - Carriers with poor Vehicle SEAs did have a slightly higher crash rate (6%) than carriers without poor Vehicle SEAs. This result is much lower than the Driver SEA, thus implying that there is a stronger relationship between driver violations and crash risk than vehicle violations.
Safety Management (SMSEA) - The Safety Management SEA is also effective in identifying carriers with high crash rates. Indicators in this SEA are based on safety regulation compliance supporting the association of safety regulations with crash risk. Carriers with high Safety Management SEAs had a 53% higher post-identification crash rate than carriers that did not have high Safety Management SEA values. Unlike the driver and vehicle SEAs, fewer carriers are assigned an SMSEA value.
Availability of SafeStat Scores
In the past, carriers with an SEA score of 75 or higher in any two areas were assigned an overall SafeStat score. However, the FMCSA has temporarily restricted public access to the (1) Accident SEA and (2) overall SafeStat score, because calculation relies on state-provided crash reports. At the present time, the agency believes reported accident data is not of the highest quality based on accuracy, completeness and timeliness. However, the Accident SEA and overall SafeStat score are planned to return to public view as soon as the agency is confident that relevant information provided by the states is more reliable.
Even though the Accident SEA and overall SafeStat score are not available for public viewing, the FMCSA and its state enforcement partners will continue to have access to all SafeStat scores to focus their safety enforcement efforts.
How Reliable is SafeStat Data?
You be the judge. The FMCSA discontinued public display of certain SafeStat scores in 2004 because data being reported (or not reported) by states was determined to skew values produced by its algorithm - the equation used to determine which carriers have safety problems.
In his July 2004 article "A Fair Move Toward Fairness", Avery Vise, Editorial Director of Commercial Carrier Journal, tackles some of the criticisms of SafeStat data.
"One of the biggest glitches in SafeStat is the disparity among states in the timeliness, completeness and accuracy of crash data reported to FMCSA," says Vise. "An audit by the Department of Transportation Office of Inspector General showed, for example, that several states had not reported any crashes for the six-month period analyzed. At least two, Pennsylvania and Florida, may have had hundreds of truck-involved crashes in that time."
In the Schramm case involving logistics giant C.H. Robinson, for example, the Maryland District Supreme Court considered SEA scores and determined SafeStat data is a reliable indicator for carrier safety measurement. As a result, Judge Motz concluded that brokers and 3PLs are responsible for checking the safety of carriers.
In March 2006, the FMCSA believed it had improved SafeStat enough to re-introduce the public to scores it had discontinued. Surrounding intense criticism, however, the agency just announced in July 2006 that it decided to stand behind its previous assessment not to display certain scores on its website, at least until findings from two new investigations are evaluated.
Today, upon accessing the SafeStat website, the FMCSA cautions visitors that "Inaccurate or out-of-date normalizing data in MCMIS can result in SafeStat results that do not accurately reflect the motor carrier's safety status." And the agency encourages users to verify the accuracy of the data prior to use. So what exactly does all of this mean?
Plenty of Room for Improvement
In a September 2004 listening session of the Comprehensive Safety Analysis 2010 initiative, participants pointed to the need for a more effective overall process to select, evaluate and rate carriers. It was suggested that compliance reviews are effective for companies that experience audits, but the number of carriers being audited (about 2%, which does not include the "mom and pop" carriers) is not sufficient. It was also mentioned that SafeStat does not identify the correct carriers for compliance reviews, and that the SafeStat formula actually penalizes growing carriers.
Other common complaints about SafeStat data were also addressed, including the fact that many carrier safety ratings are outdated, inaccurate and may be an unreliable measurement of safety compliance. As one representative said, "A satisfactory rating issued a decade ago should not be relied on to make service purchasing decisions today."
The FMCSA is focusing attention on data improvement and carrier safety, as pointed out by former Administrator Sandberg, who said "Our enforcement efforts rely heavily on high-quality data to better identify high-risk motor carriers for investigation. Our objective here is to order poor and non-compliant safety performers out of service before they cause harm. The more complete, timely and accurate our data is and the more efficient our data collection and access systems are the better our ability to prevent violations and crashes." She added that despite increasing demands, agency resources are likely to remain flat.
In her resignation letter, Sandberg noted: "During my three year tenure, we have made significant strides in improving commercial motor vehicle safety and security on our nation's highways. In this time we have reduced the agency's regulatory backlog by over 68% and provided additional enforcement focus at the local, state and federal level - resulting in the lowest large truck fatality rate since the collection of data began in 1975."
Ambitious efforts continue, according to FMCSA Assistant Administrator John H. Hill, who at the recent annual conference of the Commercial Vehicle Safety Alliance pledged to cut the large-truck fatality rate 16% by 2008 to 1.65 per 100-million miles traveled. Hill also mentioned plans to focus more on targeting problem drivers.
Additionally, the agency is working with states that have data problems, and results are improving. Now more than half the states meet FMCSA standards for providing good data quality, which is an increase from only 16 states in 2005.
Conclusion in Plain English
There are volumes of regulations, reports and other information about carrier safety. Coincidentally, there is also an absence of reliable data, and the majority of carriers remain unrated. But overall, amidst suggestions and criticisms, one simple fact remains: Published data for carrier safety "is what it is" and must be relied upon daily to make transportation decisions by everyone in the industry. And as our industry evolves, the guidelines to follow will continue to change and adapt out of necessity.
Top 10 Steps to Protect Against Hiring Unfit Carriers
1. Take a "holistic approach" when analyzing the safety fitness of your carriers in order to minimize the risk associated with negligent hiring.
2. Create or update your documented corporate policy for qualifying carriers. That policy absolutely must include consideration for all available carrier safety data.
3. Ensure your corporate policy on hiring carriers with due diligence is understood and followed by everyone involved in the carrier selection process. Do not deviate from the established policy.
4. At a minimum, your analysis process should be based on the combination of (A) carrier safety ratings (unsatisfactory, conditional, satisfactory and not rated) and (B) SafeStat SEA scores (currently driver safety and vehicle safety).
5. Maintain an up-to-date file for every carrier you use. Include information about insurance, authority, carrier safety ratings (current and previous), as well as SafeStat SEA scores. Information in these files will serve at proof of your due diligence. Carrier411 provides the ability to generate a "Due Diligence Certificate" for any motor carrier.
6. Do not hire carriers with SafeStat SEA scores greater than or equal to 75. Many companies surveyed indicated they will hire carriers with SEAs that do not exceed 72. Conversely, others said they will not hire carriers that are unrated or have been in business for less than one year. Establish your limit. When faced with evaluating a carrier that has one acceptable SEA (such as VHSEA) and another that is deemed unacceptable (such as DRSEA), you are better off classifying the carrier as unsafe or unfit.
7. Do not hire carriers with an "unsatisfactory" carrier safety rating. Determine whether you will use carriers with a "conditional" safety rating. The same rule applies to unrated carriers.
8. Monitor the safety ratings of your entire carrier base on a continual basis. React quickly and consistently to changes. The easiest and most reliable way to do this is with an automated monitoring service like Carrier411, which also provides the ability to integrate current data into any software system.
9. Supplement your research and update carrier files as needed by collecting safety related data for your stable of carriers. The more safety information you have on file about carriers, the better off you will be to support your hiring decisions - should they ever be called into question.
If a carrier has a "conditional" carrier safety rating:
(A) Call the carrier and obtain its explanation of the rating.
(B) Insist on receiving copies of the FMCSA inspection reports.
(C) Ask the carrier to explain what it is doing to change its rating to "satisfactory".
(D) Use the carrier only if you are convinced that it is sincerely trying to be in full compliance and that it probably will achieve a "satisfactory" rating.
(E) Document all this information in your file for the carrier.
If you want to use a carrier with no carrier safety rating:
(A) Conduct your own investigation before using the carrier.
(B) Research its record of accidents, FMCSA inspection reports, driver compliance with hours of service regulations, and compliance with record-keeping regulations.
(C) Ask the carrier to explain what it is doing to change its rating to "satisfactory".
(D) Document all this information in your file for the carrier.
10. Especially when qualifying new carriers, identify whether the carrier (A) previously operated under different company names and MC numbers, and (B) whether it currently operates with other MC numbers or alter egos. Evaluate the available safety data for those companies and apply your findings to the carrier being considered - as if it were one in the same.
Carriers have been known to "side step" or evade poor safety performance in the past by creating new companies and obtaining new MC numbers. In the Schramm case, for instance, Judge Motz referred to carriers with a previous existence as having "provenance" that should be recognized and evaluated.
Also in consideration of the Schramm case, your carrier contract should be updated to include protective clauses related to the decision, and every carrier used should sign your contract. Consult with a qualified transportation attorney who is familiar with the Schramm case to ensure completeness of your contract.