SIU study
Survey on performance measurement
I. Executive Summary
This study was conducted to learn how insurers measure
the performance of their SIUs. A review of the measurement systems
of 52 insurers found there is little consistency from insurer to
insurer in the methods they use in their performance systems. Findings
include:
• SIU management has a responsibility to conduct performance
review in about three of four companies with measurement systems
in place. Senior management takes a role in overseeing measurement
in approximately 42 percent of the companies.
• Most performance measurement is done annuallly (44 percent),
although nearly a third of survey participants (29 percent) said
they measure on a quarterly basis.
• The most popular method (29 percent) of calculateing dollars
not paid due to detection efforts was taking the estimated or actual
dollar amount of claims submitted. Nearly 22% of respondents said
they rely on amount of reserves to calculate savings.
• The two most cited factors used in measurement systems are
the number of referrals and quality & accuracy of investigation.
A large number also considers the training services provided by
SIUs to other departments in measuring SIUs. Insurers use an average
of five factors.
• There was almost an even split in whether insurers expensed
SIU costs on an allocated or unallocated basis.
Recommendations for the future include developing one or more models
of SIU performance to help guide insurers in structuring their programs.
A followup study also should be considered to better understand
how insurers consider deterence in their measurement systems.
II. Purpose of study
The goal of the study is to learn how insurance companies in the
United States measure the performance of their special investigation
units (SIUs), what specific measurement devices are used and how
they are applied.
Since the growth of anti-fraud efforts began in the early 1990s,
insurance companies have increasingly focused on efforts to determine
the value that SIUs bring to the corporation. Through statistical
measurement, insurers have sought to gauge effectiveness, calculate
return on investment and determine whether SIUs should be expanded,
reduced, taken in-house or contracted out. While many anti-fraud
activities were started because states required them, insurers still
seek to understand whether SIUs are cost-effective and how they
are performing over time.
A secondary purpose of this study was to determine whether there
exists enough common elements within insurer measurement systems
to suggest the creation of industry benchmarks.
A literature search did not find any data or earlier studies on
this topic.
III. Methodology
In late 2001 the Coalition Against Insurance Fraud formed a Fraud
Measurement Task Force (see roster in Appendix) and assigned this
new panel the task of studying SIU measurement, among other activities.
The Task Force developed and tested a 10-question survey that was
sent to approximately 110 SIU managers representing all lines of
insurance. The survey forms were distributed by mail in September
2002 and were returned in either hard copy or by completing a form
on the Internet. Completed survey forms were accepted through January
2003.
Survey results were tabulated and analyzed by staff of the Coalition
Against Insurance Fraud with input from the Fraud Measurement Task
Force.
IV. Survey results
A total of 52 SIU managers participated in the study by returning
survey forms and providing data about their measurement programs.
Below are the results.
- Measurement systems in place
Nearly 87 percent of insurers reported they sponsor formal programs
to measure the effectiveness of their SIUs. Of the minority that
reported no formal programs, there was no common reason why measurement
systems had not been developed.
One SIU for a large multi-line insurer wrote: “We believe
that tracking the outcome (of investigations), or dollars denied
due to fraud, creates an unacceptable and unnecessary risk of civil
litigation… In the best-case scenario, this creates the perception
that SIU investigators are compensated for denying claims and at
worst it creates actual bias by the SIU investigator.”
While some insurers may not have formal programs, they reported
they do compile statistics on anti-fraud activities for reporting
requirements mandated by many states.
Respondents who reported no measurement systems in place included
four medium-size companies, two small insurers and one large company.
- Who measures?
More than three-quarters of participants reported that the responsibility
for implementing measurement programs rests with the SIU department
itself. However, half of those respondents said that other 
departments
— mostly senior management and claims executives — also
were involved in reviewing or overseeing measurement programs. Of
the 45 respondents, 42 percent said senior management had a hand
in measuring SIU and 18 percent stated that the claims department
was involved. To a much lesser degree, legal, compliance, finance
and audit departments also were involved in measuring SIU performance.
In reviewing whether insurer size might affect which department
measures, small and large insurers tended to be measured by SIUs,
whereas medium-size insurers relied more on claims departments and
senior management to conduct measurement programs.
- How often?
Participants with measurement programs were asked how often measurement
takes place. The most common time period cited was annually (44%).
Nearly a third (29%) reported measuring unit effectiveness on a
quarterly basis. Fewer than 10% said they had no set time period
for measuring or that it was done on an on-going basis. Large and
small insurers tended to conduct reviews annually, while medium-size
insurers tended to hold quarterly reviews.
- Calculating savings
The most popular method (29 percent) for calcuting dollars not paid
due to detection 
efforts
is taking the estimated or actual dollar amount of claims submitted.
Nearly 22% said they rely on amount of reserves to calculate savings.
Another 24% said they use reserves and the number of closed claims.
Six SIU managers reported they used other methods or intentionally
did not calculate savings. There were no major deviations in reviewing
respondent data by insurer size.
- Factors in measuring performance
Respondents were presented with 15 possible factors that might be
used in rating SIU performance. Only two factors were cited by a
majority of respondents — number of referrals and quality
and accuracy of investigations. How the latter factor is defined
and determined was not explored since evaluating quality can be
nebulous and intangible.
All but two respondents used the number of referrals their SIUs
receive as a factor. Related to this factor is the percentage of
claims that is referred to the SIU, a factor that is used by nearly
half (49.8%) of respondents. The next most-used factor (46.7%) was
the number of claims files referred to SIU that were closed without
payment. 
The accompanying graph shows which factors insurers use, and how
much. There also was a wide distribution in the number of factors
used by the respondents. Some companies only use one factor in the
list of 15, while others use as many as 14. The average number of
factors used in measurement systems was 5.4.
- Other factors
Respondents were asked whether their SIUs were measured on non-detection
activities, including their actions to deter future fraudulent acts,
fraud training and other activities 
not
traditionally related to fraud. A large majority considered training
activities performed by SIUs in their measurement systems. Such
training usually includes education of claims, underwriting and
other internal departments.
The value of deterrence is used by a quarter of respondents in their
measurement systems. A third of respondents also used factors not
related to fraud. As a group, fewer large insurers (19%) included
non-fraud activities in their measurement program than medium-size
(50%) and small insurers (40%).
- Expensing SIU costs
Understanding how insurers account for the costs of their SIUs is
important because those cost measurements are sometimes used in
conjunction with SIU savings to determine whether anti-fraud activities
truly add to the insurer’s bottom line. Whether expenses should
be allocated to the claims file or unallocated as an administrative
expense is a discussion beyond this study, but is a common topic
of discussion within SIUs and insurance companies. The respondents
are roughly equally split in the method they use (44% vs. 46%),
with the remainder using a combination of the two methods. Small
insurers tended to use allocated systems while medium-size insurers
tended to use unallocated systems.
- Areas of insurance investigated
Slightly more than eight of 10 respondents investigated suspected
fraud involving automobile claims, the most cited area of investigation
by respondents. Other areas cited by the majority of respondents
included homeowners insurance, agent & broker fraud, commercial
liability and internal fraud by company employees. Investigators
of health insurance fraud represented the smallest percentage of
respondents in the study.
- Size of participating insurance companies
The study solicited responses from a variety of types and sizes
of insurers to provide a fair cross-section of the insurance industry.
Large insurers represented approximately 44 percent of respondents,
medium-size insurance 32 percent and small insurance 24 percent.
V. Conclusions
Insurers use a variety of systems to measure their SIUs, with little
commonality among factors used. Few correlations seem to exist between
the types of systems and the size or line of business. That is,
similar insurance companies used dissimilar systems. There is even
great disparity in how often they measure performance. The prospect
of developing an industry-wide system to help establish benchmarks
for SIU performance seems bleak unless many insurers are willing
to change their measurement programs.
However, insurers and others interested in promoting effective SIU
measurement systems should consider developing model programs that
would be helpful, especially for insurance companies that are just
beginning to measure anti-fraud activity.
Insurers appear to fall into three broad, sometimes overlapping,
categories: those without measurement systems, those that have them
because states mandate annual data on anti-fraud activities, and
those that see such systems as a true management tool to help to
guide effectiveness.
Insurers that don’t measure SIUs apparently do not conduct
business in those states that require it, have relatively new anti-fraud
programs or feel that measurement systems could be used against
them in bad-faith litigation.
Three of four respondents report that measurement is conducted by
SIU. Some may be concerned about such a high percentage conducting
what appears to be a self-evaluation, which may not be as credible
as one done by senior management or an “independent”
department such as corporate audit. However, a closer analysis of
the data reveals that half of those respondents measure performance
in conjunction with another department, most likely in the review
process. Still, that leaves half of those insurers — and 40%
of all respondents — where SIUs alone are measuring their
own operations.
Calculating savings from anti-fraud activities is another area where
there is little commonality. Most either use the amount of the claim
submitted or the reserve, but how companies arrive at that latter
determination can vary greatly.
One clear finding in reviewing non-detection activities as measurement
factors is the large number of SIUs that are involved in training
internal departments about fraud. The fact that 84 percent consider
training as a factor indicates the increasing emphasis on importance
of having claims, underwriting and other departments knowledgeable
about fraud and how to detect it or prevent it.
A quarter of respondents also said that deterrence is a factor in
their measurement systems. Since measuring deterrence itself is
difficult, a followup study may be needed to further explore how
these insurers are using deterrence as a measurement factor.
No absolute conclusions should be derived from such a small sample
of insurance companies, even though this sample represents a significant
portion of the property/casualty industry marketshare. However,
the authors encouarge insurers interested in enhancing their anti-fraud
operations to review these findings with an eye on how they might
refine their operations to have the greatest impact on preventing
and detecting insurance fraud.
Click here to view a copy of the survey instrument
Participating insurers
Accident Fund Insurance Company of America
ACE USA
Allmerica Financial
Allstate
American Express Property Casualty
American General
American National Property & Casualty
Amerisure Mutual Insurance Company
Amica
Bristol West
California Casualty
Celina Insurance Group
Commerce Insurance Company
Countryway Insurance Company
Crum & Forster Insurance
Empire Fire and Marine
Erie
Farmers Insurance
Fireman's Fund
Geico
GMAC Insurance
Great West Casualty
Hartford Life
John Hancock Life Insurance Company
Kemper Insurance
Legion
MAMSI
Merchants Insurance
Meridian Resource Company
MetLife Auto & Home
MSI Insurance
One Beacon
Pemco Insurance
Penn National
Progressive
Prudential Property & Casualty
Robert Plan
Safeco
SC Farm Bureau
Scottsdale Insurance
Shelter Insurance
St. Paul
State Farm
The Hartford
Unigard Insurance
United Healthcare
Unitrin Direct Auto Insurance
Universal Underwriters Group
Unum Provident
Vesta Insurance
Wellpoint Health Networks
Fraud Measurement Task Force
Jim Brown
Center for Consumer Affairs, University of Wisconsin/Milwaukee
Gerry Lynch
ChoicePoint
Jim Bonk
CNA
Brian Soltis
Fireman's Fund
Joseph J. Cohen
International Assn of Insurance Fraud Agencies
Tommy Short
International Assn of SIUs
Seth Perlmutter
LexisNexis
Dennis Toomey
Liberty Mutual
Alan Fridkin
MassMutual
Kevin O’Connor
MJM Investigations
Ray Albertini
Progressive Insurance
Dennis Schulkins
State Farm
Kent Davis
St. Paul Insurance
Jack Houston
Travelers Insurance
Staff
Dennis Jay
Coalition Against Insurance Fraud
