Studies that performed sentiment analysis using supervised approach, including the number of raters and associated inter-rater agreement expressed as Cohen’s kappa (κ), classifiers and configuration applied where reported. Studies are reported in chronological order
| Author | Data source | Comments classified | No. of raters | κ | Sentiment categories | Classifier | Configuration | |||||||||
| Positive | Negative | Mixed | Neutral | SVM | NB | DT | B | RF | GL | KN | ||||||
| Alemi et al34 | RateMDs | 100%* (n=955) | NR | NR | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Sparsity rule Information gain SVM: RBF kernel | |||
| Greaves et al7 | NHS choices | 17.56%† (1000/5695) | 2 | 0.76 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Prior polarity Information gain SVM: RBF kernel | |||
| Wagland et al48 | Cancer experience | 14.19% (800/5634) | 3 | 0.64–0.87 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | NR | |||||
| Bahja et al26 | NHS choices | 75% (56 818/76 151) | N/A | N/A | ✓ | ✓ | ✓ | ✓ | Sparsity rule Ratings in binary sentiment | |||||||
| Jimenez-Zafra et al54 | COPOS and COPOD‡ | 100% (n=156 975 COPOD and n=743 COPOS) | N/A | N/A | ✓ | ✓ | ✓ | Ratings in binary sentiment SVM: linear kernel | ||||||||
| Huppertz et al6 | 0.88% (508/57 986) | 3 | NR | ✓ | ✓ | ✓ | ✓ | ✓ | NR | |||||||
| Doing-Harris et al24 | Press Ganey | 0.58% (300/51 235) | 3 | 0.73 | ✓ | ✓ | ✓ | NR | ||||||||
| Menendez et al47 | Vendor supplied | 100% (132/132) | NR | NR | ✓ | ✓ | ✓ | ✓ | NR | |||||||
*Classified as praise (positive), complaint (negative), praise and complaint (mixed), neither (neutral).
†Only n-grams classified.
‡Also used dictionary lookup and cross domain method.
B, bagging; COPOD, corpus of patient opinions in Dutch; COPOS, corpus of patient opinions in Spanish; DT, decision trees; GL, generalised linear model; KN, k-nearest neighbour; NB, Naïve Bayes; NR, not reported; RBF, radial basis function; RF, random forest; SVM, support vector machine.