Rutger Daniel van Oest

Oest, Rutger Daniel van (2022)

The Dependence of Chance-Corrected Weighted Agreement Coefficients on the Power Parameter of the Weighting Scheme: Analysis and Measurement

Psychometrika Doi: 10.1007/s11336-022-09881-7

Empirical studies that investigate the effect of design thinking within complex contexts involving multiple stakeholders are rare. The aim of this study is to contribute to the literature on design thinking, by investigating the perceived usefulness of including design thinking activities into a complex research project for food safety. A survey was distributed to all participants in SafeConsume, a Horizon 2020 research project, to measure perceived usefulness of design thinking activities such as collaborative workshops, visualization tools and empathic observation studies. Bivariate correlations and one-way ANOVAs were conducted in JMP Pro 14. The results indicate that design thinking activities may be useful also for large food safety projects. Multidisciplinary collaborative workshops can generate optimism and a sense of belonging among the participants, visualization tools can contribute to simplify complex information, and empathic observation studies makes it easier to think user centric. This study is one of few that quantitatively investigate the perceived usefulness of implementing design thinking into a multidisciplinary research project, and the findings contribute to a better understanding of the perceived effects of implementing design thinking into a large complex food safety research projects.

van Oest, Rutger & Girard, Jeffrey M. (2021)

Weighting Schemes and Incomplete Data: A Generalized Bayesian Framework for Chance-Corrected Interrater Agreement

Psychological methods Doi: 10.1037/met0000412 - Full text in research archive

Van Oest (2019) developed a framework to assess interrater agreement for nominal categories and complete data. We generalize this framework to all four situations of nominal or ordinal categories and complete or incomplete data. The mathematical solution yields a chance-corrected agreement coefficient that accommodates any weighting scheme for penalizing rater disagreements and any number of raters and categories. By incorporating Bayesian estimates of the category proportions, the generalized coefficient also captures situations in which raters classify only subsets of items; that is, incomplete data. Furthermore, this coefficient encompasses existing chance-corrected agreement coefficients: the S-coefficient, Scott’s pi, Fleiss’ kappa, and Van Oest’s uniform prior coefficient, all augmented with a weighting scheme and the option of incomplete data. We use simulation to compare these nested coefficients. The uniform prior coefficient tends to perform best, in particular, if one category has a much larger proportion than others. The gap with Scott’s pi and Fleiss’ kappa widens if the weighting scheme becomes more lenient to small disagreements and often if more item classifications are missing; missingness biases play a moderating role. The uniform prior coefficient often performs much better than the S-coefficient, but the S-coefficient sometimes performs best for small samples, missing data, and lenient weighting schemes. The generalized framework implies a new interpretation of chance-corrected weighted agreement coefficients: These coefficients estimate the probability that both raters in a pair assign an item to its correct category without guessing. Whereas Van Oest showed this interpretation for unweighted agreement, we generalize to weighted agreement.

van Oest, Rutger Daniel (2019)

Unconstrained Cholesky-based parametrization of correlation matrices

Communications in Statistics - Simulation and Computation Doi: 10.1080/03610918.2019.1628271 - Full text in research archive

Parameter estimation is relatively complicated for models containing correlation matrices, because the elements of correlation matrices are heavily constrained. We put forward a Cholesky-based parametrization that is easy to implement and allows for unconstrained parameter estimation. To compare the new parametrization with the commonly applied spherical parametrization, we use Monte Carlo simulation in which we estimate multivariate distributions containing Gaussian copulas. We show that the new parametrization performs well, in particular as the dimensionality of the multivariate distribution increases, computing times increase, and non-convergence occurs increasingly often.

van Oest, Rutger Daniel (2019)

A New Coefficient of Interrater Agreement: The Challenge of Highly Unequal Category Proportions

Psychological methods, 24(4), s. 439- 451. Doi: 10.1037/met0000183 - Full text in research archive

We derive a general structure that encompasses important coefficients of interrater agreement such as the S-coefficient, Cohen’s kappa, Scott’s pi, Fleiss’ kappa, Krippendorff’s alpha, and Gwet’s AC1. We show that these coefficients share the same set of assumptions about rater behavior; they only differ in how the unobserved category proportions are estimated. We incorporate Bayesian estimates of the category proportions and propose a new agreement coefficient with uniform prior beliefs. To correct for guessing in the process of item classification, the new coefficient emphasizes equal category probabilities if the observed frequencies are unstable due to a small sample, and the frequencies increasingly shape the coefficient as they become more stable. The proposed coefficient coincides with the S-coefficient for the hypothetical case of zero items; it converges to Scott’s pi, Fleiss’ kappa, and Krippendorff’s alpha as the number of items increases. We use simulation to show that the proposed coefficient is as good as extant coefficients if the category proportions are equal and that it performs better if the category proportions are substantially unequal.

Andreassen, Tor W.; van Oest, Rutger Daniel & Lervik-Olsen, Line (2018)

Customer Inconvenience and Price Compensation: A Multiperiod Approach to Labor-Automation Trade-Offs in Services

Journal of Service Research, 21(2), s. 173- 183. Doi: 10.1177/1094670517738370

Koulaei, Afra; van Oest, Rutger Daniel & Bharadwaj, Sundar (2019)

Familiarity Mitigates the Effect of Team Reward on Functional Conflict and Citizenship Behavior

[Academic lecture]. Academy of Management Annual Meeting.

Ungureanu, Delia Olga & van Oest, Rutger Daniel (2018)

When Customer Spending Backfires: Customer Trade-offs Between Durable Products Replacements and Add-on Purchases

[Academic lecture]. EMAC Conference.

Ungureanu, Delia Olga & van Oest, Rutger Daniel (2018)

When Customer Spending Backfires: Customer Trade-offs Between Durable Products Replacements and Add-on Purchases

[Academic lecture]. Marketing Science Conference.

van Oest, Rutger Daniel & Knox, George (2018)

Incorporating Satisfaction Surveys Into Customer Base Analysis

[Academic lecture]. EMAC 2018.

van Oest, Rutger Daniel (2017)

Inter-rater Reliability: The Challenge of Highly Unequal Category Sizes

[Academic lecture]. 2017 Marketing Science Conference.