Comparison of ordinal and nominal classification trees to predict ordinal expert-based occupational exposure estimates in a case-control study.

Abstract

OBJECTIVES:

To evaluate occupational exposures in case-control studies, exposure assessors typically review each job individually to assign exposure estimates. This process lacks transparency and does not provide a mechanism for recreating the decision rules in other studies. In our previous work, nominal (unordered categorical) classification trees (CTs) generally successfully predicted expert-assessed ordinal exposure estimates (i.e. none, low, medium, high) derived from occupational questionnaire responses, but room for improvement remained. Our objective was to determine if using recently developed ordinal CTs would improve the performance of nominal trees in predicting ordinal occupational diesel exhaust exposure estimates in a case-control study.

METHODS:

We used one nominal and four ordinal CT methods to predict expert-assessed probability, intensity, and frequency estimates of occupational diesel exhaust exposure (each categorized as none, low, medium, or high) derived from questionnaire responses for the 14983 jobs in the New England Bladder Cancer Study. To replicate the common use of a single tree, we applied each method to a single sample of 70% of the jobs, using 15% to test and 15% to validate each method. To characterize variability in performance, we conducted a resampling analysis that repeated the sample draws 100 times. We evaluated agreement between the tree predictions and expert estimates using Somers' d, which measures differences in terms of ordinal association between predicted and observed scores and can be interpreted similarly to a correlation coefficient.

RESULTS:

From the resampling analysis, compared with the nominal tree, an ordinal CT method that used a quadratic misclassification function and controlled tree size based on total misclassification cost had a slightly better predictive performance that was statistically significant for the frequency metric (Somers' d: nominal tree = 0.61; ordinal tree = 0.63) and similar performance for the probability (nominal = 0.65; ordinal = 0.66) and intensity (nominal = 0.65; ordinal = 0.65) metrics. The best ordinal CT predicted fewer cases of large disagreement with the expert assessments (i.e. no exposure predicted for a job with high exposure and vice versa) compared with the nominal tree across all of the exposure metrics. For example, the percent of jobs with expert-assigned high intensity of exposure that the model predicted as no exposure was 29% for the nominal tree and 22% for the best ordinal tree.

CONCLUSIONS:

The overall agreements were similar across CT models; however, the use of ordinal models reduced the magnitude of the discrepancy when disagreements occurred. As the best performing model can vary by situation, researchers should consider evaluating multiple CT methods to maximize the predictive performance within their data.

• Authors: David C Wheeler, Kellie J Archer, Igor Burstyn, Kai Yu, Patricia A Stewart, Joanne S Colt, Dalsu Baris, Margaret R Karagas, Molly Schwenn, Alison Johnson, Karla Armenti, Debra T Silverman, Melissa C Friesen
• Journal: The Annals of occupational hygiene (Ann Occup Hyg) Vol. 59 Issue 3 Pg. 324-35 (Apr 2015) ISSN: 1475-3162 [Electronic] England
• PMID: 25433003 (Publication Type: Comparative Study, Journal Article, Research Support, N.I.H., Extramural)
• Copyright: © The Author 2014. Published by Oxford University Press on behalf of the British Occupational Hygiene Society.

Chemical References

• Air Pollutants, Occupational
• Vehicle Emissions

Topics

• Air Pollutants, Occupational (analysis)
• Algorithms
• Case-Control Studies
• Decision Support Techniques
• Humans
• Male
• Models, Statistical
• Occupational Exposure (adverse effects, analysis)
• Research Design
• Surveys and Questionnaires
• Vehicle Emissions (analysis)