roc.area {verification} | R Documentation |
This function calculates the area underneath a ROC curve following the process outlined in Mason and Graham (2002). The p-value produced is related to the Mann-Whitney U statistics. The p-value is calculated using the wilcox.test function which automatically handles ties and makes approximations for large values.
The p-value addresses the null hypothesis $H_o:$ The area under the ROC curve is 0.5 i.e. the forecast has no skill.
roc.area(obs, pred)
obs |
A binary observation (coded {0, 1 } ). |
pred |
A probability prediction on the interval [0,1]. |
A |
Area under ROC curve, adjusted for ties in forecasts, if present |
n.total |
Total number of records |
n.events |
Number of events |
n.noevents |
Number of non-events |
p.value |
Unadjusted p-value |
This function is used internally in the roc.plot
command
to calculate areas.
Matt Pocernich <pocernic@rap.ucar.edu>
Mason, S.J. and N.E. Graham. (2002) “Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation, ” Q. J. R. Meteorol. Soc. 30 (1982) 291-303.
# Data used from Mason and Graham (2002). a<- c(1981, 1982, 1983, 1984, 1985, 1986, 1987, 1988, 1989, 1990, 1991, 1992, 1993, 1994, 1995) b<- c(0,0,0,1,1,1,0,1,1,0,0,0,0,1,1) c<- c(.8, .8, 0, 1,1,.6, .4, .8, 0, 0, .2, 0, 0, 1,1) d<- c(.928,.576, .008, .944, .832, .816, .136, .584, .032, .016, .28, .024, 0, .984, .952) A<- data.frame(a,b,c, d) names(A)<- c("year", "event", "p1", "p2") ## for model with ties roc.area(A$event, A$p1) ## for model without ties roc.area(A$event, A$p2)