The validity of the assessment centre in predicting managerial performance of business development managers

Abstract

Having recognised the need to keep abreast and meeting the demands of the changing business environment, a well-known insurance company in South Africa (referred to as “the Company”), decided in October 2000 to restructure the advisers’ channel within its sales business unit. Several major challenges, such as growth in focused market share, growth in premium income, growth in volume, profitable distribution, and a high-performance culture, had to be addressed. All Regional Managers, Branch Managers and Assistant Branch Managers within the Company were affected by this business decision. After the consultation process had been finalised, 183 employees were given the opportunity to apply for 86 Business Development Manager positions nationally. The selection criteria rested on a competency-based evaluation process. Each candidate attended a one-day assessment centre, comprising of a leaderless group discussion exercise, an in-basket, a coaching interview and a written case study, measuring 13 different competencies. Specific production standards with reference to growth in premium income, growth in volume and growth in manpower were also considered as part of the selection criteria. Irrespective of these appointments, a number of black Business Development Managers from the external market, were also appointed to the Company on a national basis since June 2001, due to further expansions in the black upcoming market. The total number of Business Development Managers appointed by the end of December 2002, amounted to 116. The rationale behind the position of Business Development Manager is to become more creative in order to generate business, add value and ensure organisational success. The ability to convert ideas into business within a sales franchise environment, is an important requirement for a Business Development Manager. This will result in increased volumes, growth and market share which is critical for the survival of the Company. A team of 15-20 Advisers supports the Business Development Manager to achieve these business objectives. The recently passed Employment Equity Act 1998 states clearly that psychometric testing and other similar assessments of an employee are prohibited, unless proven to be scientifically valid, fairly applicable to all employees, and not biased towards any employee or group. Employers must therefore, subject themselves to situation specific research with reference to psychometric properties of assessment instruments, where applicable. No conclusive evidence is currently available to indicate that Business Development Managers, appointed since January 2001, by means of competency-based assessment procedures, are successful and meet their respective business targets. The purpose of this study, therefore, is to establish the predictive validity of a managerial assessment battery for Business Development Managers within the Company. This will determine whether the competencies identified, and the assessment instruments used during the evaluation process, is projecting a clear indication of managerial potential amongst Business Development Managers, as well as the successful achievement of business results. The vital question regarding specific competencies essential in appointing successful Business Development Managers, will be identified and well researched. Furthermore, evidence of the predictive validity of assessment instruments, is essential in determining the level of accuracy with which work-related behaviour is predicted. The strength of the statistical relationship between a predictor score (the assessment centre results) and a criterion score (the work-performance results) indicates the predictive validity of the measure. The sample for this research study consisted of a convenience sample of 92 managers, who participated in an assessment centre evaluation for managerial competencies, during the period 2000 to 2002. Assessment centre ratings were measured against three different variables. The first variable was a business indicator known as Score point, calculated for each Business Development Manager as on 31 December 2002. This performance outcome indicator is used as an internal recognition system where intermediaries score points, based on the volume of business they generated during a specific period. Another business indicator known as Weighted point, which is also an internal measurement system, where different elements of a Business Development Manager’s performance is measured and weighted, was calculated for each Business Development Manager as on 31 December 2002. In November 2002 performance ratings were conducted by each Provincial Manager, evaluating his/her team of Business Development Managers respectively, which was seen as the third variable for the purpose of this study. This evaluation was based on several behavioural competencies as demonstrated by the Business Development Manager in his/her work environment. These 13 competencies were the same 13 dimensions measured in the managerial assessment centre, conducted during the selection process. For the purpose of this research report, “Score” or “Score point” refers to a performance outcome indicator, used as an internal recognition system in this particular insurance company, whereas “score” refers to the result(s) obtained by a candidate, applying for a Business Development Manager position in the Company, who attends a managerial assessment centre as part of the selection process. Biographical and other company-related information was collected for each Business Development Manager and captured, together with assessment centre data and performance data. The data was screened and statistically analysed using descriptive statistics, correlation coefficients, factor analysis and multiple regression analysis. Based on the sample, the Coastal region was statistically the best represented sub-group at 33%. The remaining percentage (67%) of cases was more or less equally distributed amongst the other regions. The white group constituted 85% of the sample, followed by the black group at 12%. The coloured and Indian groups were least represented, at 2% and 1% respectively. Male Business Development Managers, representing 97% of the sample, dominated the sample. The age of the sample group ranged between 37 and 62 years, with an average age of 46 years. The majority of the Business Development Managers (85%) were appointed on 1 January 2001. The rural market group (32%) was the best represented sub-group in the sample, followed by the self-employed group (28%), the salaried group (24%) and the emerging market group (16%). The results obtained in the study, compared well to the results of similar studies conducted elsewhere. The validity of the assessment centre varied between low to moderate and high in predicting managerial performance in accordance with the Weighted point (R = 0.251), Score point (R = 0.414) and behavioural performance (R = 0.499). It can therefore be concluded, that the predictive validity of the assessment centre is superior in terms of behaviour related managerial performance, compared to organisational outcome variables. These results are consistent with what is to be expected from behaviourally based assessment instruments. A statistical significant multiple correlation of 0.519 was also obtained between the supervisor’s rating of behavioural performance and the Score point, signifying that performance in terms of work behaviour, relates strongly to tangible performance outcomes as well. This finding indicates that high managerial ratings, correlates with a high Score point. The best predictors of actual performance appear to be Action Orientation and Financial Management Skills, followed by Judgement and Decisiveness, Leadership Skills and Business Strategy Skills. However, the best predictors of managerial behavioural performance appear to be Self-Motivation, Performance Management Skills, and Judgement and Decisiveness, followed by Action Orientation, Entrepreneurship, Leadership Skills, Business Operation Skills, Business Strategy Skills and Energy. The best predictors of the Weighted point are Action Orientation, and Judgement and Decisiveness. Another interesting conclusion is that both Action Orientation, and Judgement and Decisiveness, appear to be very good predictors of actual performance, managerial behavioural performance and the Weighted point. Factor analyses revealed that the behavioural competencies included in the managerial assessment centre, could be divided in two main components. The first and most important component can broadly be defined as embedded entrepreneurial and leadership qualities. The component consists of Entrepreneurship, Leadership Skills, Coaching Skills, Action Orientation, Judgement and Decisiveness, Assertiveness, Self-motivation and Energy. The second component can be described as essential qualities for effective day-to-day business management and consists of Performance Management Skills, Business Operation Skills, Business Strategy Skills, Financial Management Skills and Adaptability. The two components explain a large proportion of the differences in behavioural competencies that were observed for Business Development Managers. The results of the factor analysis performed on the supervisor’s rating of behavioural performance, were very similar to the assessment centre results. The two main components were largely replicated and indicate that the behavioural competencies and their interrelationships are well understood and manifested in a consistent manner, for both the assessment centre evaluations and supervisor performance ratings. The results therefore, revealed that the supervisor’s rating of performance is projecting a clear indication of the overall performance of the Business Development Managers. The evidence acquired by means of factor analysis, provides additional credibility to the findings of the study. A limiting factor in validity studies is the difficulty to determine the extent to which performance ratings are biased. The same applies to this study. Should behavioural performance ratings in future form part of the performance ratings of Business Development Managers, Provincial Managers must be sensitised to the limiting effect of bias. The effect of central tendency and/or the selection ratio also appears to limit the score variance of the assessment centre, which jeopardises the discrimination value of the Provincial Managers’ rating of behavioural performance. The use of a seven-point rating scale instead of a five-point rating scale could be considered to provide more scope for score variation, and consequently, result in better discrimination of performance levels. Another limitation is that the Weighted point was characterised by a large number of extreme scores, where the distribution deviated significantly from the normal distribution. It is recommended that the process to which the Weighted point is compiled, be reviewed prior to future use in similar studies. A further suggestion is, that by using assessment centre scores in combination with objective tests and structured interviewing techniques, the validity of the selection battery, used by the Company in appointing Business Development Managers, is expected to increase significantly. Proven dimensions identified in the study, must be evaluated and re-defined in the form of behavioural indicators, and weights allocated for each dimension in terms of importance. These well-proven dimensions must continually be monitored and rewarded in performance management, and be linked to formal and informal development initiatives. The assessment of behavioural performance dimensions should also be included as part of the routine assessment of Business Development Managers, as these dimensions could contribute significantly to an understanding of the effectiveness of behaviourally based assessments, as well as that of the relationship between behavioural performance and tangible performance outcomes. In conclusion, the managerial assessment centre appears to have significant predictive validity. Future performance of individuals, can be predicted substantially more accurately, when applying the results of the assessment centre as part of the selection process, compared to not utilising the assessment centre at all. Research evidence suggests that, when applied correctly, assessment centre evaluations can contribute significantly to effective employment decisions. The application of the assessment centre could result in substantial benefits for this insurance company, in respect of increased productivity and reduced employment costs.