HypoTestResult is a base class for results from hypothesis tests.
Any tool inheriting from HypoTestCalculator can return a HypoTestResult. As such, it stores a p-value for the null-hypothesis (eg. background-only) and an alternate hypothesis (eg. signal+background). The p-values can also be transformed into confidence levels ( \(CL_{b}\), \(CL_{s+b}\)) in a trivial way. The ratio of the \(CL_{s+b}\) to \(CL_{b}\) is often called \(CL_{s}\), and is considered useful, though it is not a probability. Finally, the p-value of the null can be transformed into a number of equivalent Gaussian sigma using the Significance method.
The p-value of the null for a given test statistic is rigorously defined and this is the starting point for the following conventions.
The p-value for the null and alternate are on the same side of the observed value of the test statistic. This is the more standard convention and avoids confusion when doing inverted tests.
For exclusion, we also want the formula \(CL_{s} = CL_{s+b} / CL_{b}\) to hold which therefore defines our conventions for \(CL_{s+b}\) and \(CL_{b}\). \(CL_{s}\) was specifically invented for exclusion and therefore all quantities need be related through the assignments as they are for exclusion: \(CL_{s+b} = p_{s+b}\); \(CL_{b} = p_{b}\). This is derived by considering the scenarios of a powerful and not powerful inverted test, where for the not so powerful test, \(CL_{s}\) must be close to one.
For results of Hypothesis tests, \(CL_{s}\) has no similar direct interpretation as for exclusion and can be larger than one.
Definition at line 22 of file HypoTestResult.h.
Public Member Functions | |
HypoTestResult (const char *name, Double_t nullp, Double_t altp) | |
constructor from name, null and alternate p values | |
HypoTestResult (const char *name=0) | |
default constructor | |
HypoTestResult (const HypoTestResult &other) | |
copy constructor | |
virtual | ~HypoTestResult () |
destructor | |
virtual Double_t | AlternatePValue () const |
Return p-value for alternate hypothesis. | |
virtual void | Append (const HypoTestResult *other) |
add values from another HypoTestResult | |
virtual Double_t | CLb () const |
Convert NullPValue into a "confidence level". | |
Double_t | CLbError () const |
The error on the "confidence level" of the null hypothesis. | |
virtual Double_t | CLs () const |
\(CL_{s}\) is simply \(CL_{s+b}/CL_{b}\) (not a method, but a quantity) | |
Double_t | CLsError () const |
The error on the ratio \(CL_{s+b}/CL_{b}\). | |
virtual Double_t | CLsplusb () const |
Convert AlternatePValue into a "confidence level". | |
Double_t | CLsplusbError () const |
The error on the "confidence level" of the alternative hypothesis. | |
const RooArgList * | GetAllTestStatisticsData (void) const |
RooDataSet * | GetAltDetailedOutput (void) const |
SamplingDistribution * | GetAltDistribution (void) const |
Bool_t | GetBackGroundIsAlt (void) const |
RooDataSet * | GetFitInfo () const |
RooDataSet * | GetNullDetailedOutput (void) const |
SamplingDistribution * | GetNullDistribution (void) const |
Bool_t | GetPValueIsRightTail (void) const |
Double_t | GetTestStatisticData (void) const |
Bool_t | HasTestStatisticData (void) const |
virtual Double_t | NullPValue () const |
Return p-value for null hypothesis. | |
Double_t | NullPValueError () const |
The error on the Null p-value. | |
HypoTestResult & | operator= (const HypoTestResult &other) |
assignment operator | |
void | Print (const Option_t *="") const |
Print out some information about the results Note: use Alt/Null labels for the hypotheses here as the Null might be the s+b hypothesis. | |
void | SetAllTestStatisticsData (const RooArgList *tsd) |
void | SetAltDetailedOutput (RooDataSet *d) |
void | SetAltDistribution (SamplingDistribution *alt) |
void | SetBackgroundAsAlt (Bool_t l=kTRUE) |
void | SetFitInfo (RooDataSet *d) |
void | SetNullDetailedOutput (RooDataSet *d) |
void | SetNullDistribution (SamplingDistribution *null) |
void | SetPValueIsRightTail (Bool_t pr) |
void | SetTestStatisticData (const Double_t tsd) |
virtual Double_t | Significance () const |
familiar name for the Null p-value in terms of 1-sided Gaussian significance | |
Double_t | SignificanceError () const |
The error on the significance, computed from NullPValueError via error propagation. | |
Public Member Functions inherited from TNamed | |
TNamed () | |
TNamed (const char *name, const char *title) | |
TNamed (const TNamed &named) | |
TNamed copy ctor. | |
TNamed (const TString &name, const TString &title) | |
virtual | ~TNamed () |
TNamed destructor. | |
virtual void | Clear (Option_t *option="") |
Set name and title to empty strings (""). | |
virtual TObject * | Clone (const char *newname="") const |
Make a clone of an object using the Streamer facility. | |
virtual Int_t | Compare (const TObject *obj) const |
Compare two TNamed objects. | |
virtual void | Copy (TObject &named) const |
Copy this to obj. | |
virtual void | FillBuffer (char *&buffer) |
Encode TNamed into output buffer. | |
virtual const char * | GetName () const |
Returns name of object. | |
virtual const char * | GetTitle () const |
Returns title of object. | |
virtual ULong_t | Hash () const |
Return hash value for this object. | |
virtual Bool_t | IsSortable () const |
virtual void | ls (Option_t *option="") const |
List TNamed name and title. | |
TNamed & | operator= (const TNamed &rhs) |
TNamed assignment operator. | |
virtual void | Print (Option_t *option="") const |
Print TNamed name and title. | |
virtual void | SetName (const char *name) |
Set the name of the TNamed. | |
virtual void | SetNameTitle (const char *name, const char *title) |
Set all the TNamed parameters (name and title). | |
virtual void | SetTitle (const char *title="") |
Set the title of the TNamed. | |
virtual Int_t | Sizeof () const |
Return size of the TNamed part of the TObject. | |
Public Member Functions inherited from TObject | |
TObject () | |
TObject constructor. | |
TObject (const TObject &object) | |
TObject copy ctor. | |
virtual | ~TObject () |
TObject destructor. | |
void | AbstractMethod (const char *method) const |
Use this method to implement an "abstract" method that you don't want to leave purely abstract. | |
virtual void | AppendPad (Option_t *option="") |
Append graphics object to current pad. | |
virtual void | Browse (TBrowser *b) |
Browse object. May be overridden for another default action. | |
ULong_t | CheckedHash () |
Check and record whether this class has a consistent Hash/RecursiveRemove setup (*) and then return the regular Hash value for this object. | |
virtual const char * | ClassName () const |
Returns name of class to which the object belongs. | |
virtual void | Delete (Option_t *option="") |
Delete this object. | |
virtual Int_t | DistancetoPrimitive (Int_t px, Int_t py) |
Computes distance from point (px,py) to the object. | |
virtual void | Draw (Option_t *option="") |
Default Draw method for all objects. | |
virtual void | DrawClass () const |
Draw class inheritance tree of the class to which this object belongs. | |
virtual TObject * | DrawClone (Option_t *option="") const |
Draw a clone of this object in the current selected pad for instance with: gROOT->SetSelectedPad(gPad) . | |
virtual void | Dump () const |
Dump contents of object on stdout. | |
virtual void | Error (const char *method, const char *msgfmt,...) const |
Issue error message. | |
virtual void | Execute (const char *method, const char *params, Int_t *error=0) |
Execute method on this object with the given parameter string, e.g. | |
virtual void | Execute (TMethod *method, TObjArray *params, Int_t *error=0) |
Execute method on this object with parameters stored in the TObjArray. | |
virtual void | ExecuteEvent (Int_t event, Int_t px, Int_t py) |
Execute action corresponding to an event at (px,py). | |
virtual void | Fatal (const char *method, const char *msgfmt,...) const |
Issue fatal error message. | |
virtual TObject * | FindObject (const char *name) const |
Must be redefined in derived classes. | |
virtual TObject * | FindObject (const TObject *obj) const |
Must be redefined in derived classes. | |
virtual Option_t * | GetDrawOption () const |
Get option used by the graphics system to draw this object. | |
virtual const char * | GetIconName () const |
Returns mime type name of object. | |
virtual char * | GetObjectInfo (Int_t px, Int_t py) const |
Returns string containing info about the object at position (px,py). | |
virtual Option_t * | GetOption () const |
virtual UInt_t | GetUniqueID () const |
Return the unique object id. | |
virtual Bool_t | HandleTimer (TTimer *timer) |
Execute action in response of a timer timing out. | |
Bool_t | HasInconsistentHash () const |
Return true is the type of this object is known to have an inconsistent setup for Hash and RecursiveRemove (i.e. | |
virtual void | Info (const char *method, const char *msgfmt,...) const |
Issue info message. | |
virtual Bool_t | InheritsFrom (const char *classname) const |
Returns kTRUE if object inherits from class "classname". | |
virtual Bool_t | InheritsFrom (const TClass *cl) const |
Returns kTRUE if object inherits from TClass cl. | |
virtual void | Inspect () const |
Dump contents of this object in a graphics canvas. | |
void | InvertBit (UInt_t f) |
Bool_t | IsDestructed () const |
IsDestructed. | |
virtual Bool_t | IsEqual (const TObject *obj) const |
Default equal comparison (objects are equal if they have the same address in memory). | |
virtual Bool_t | IsFolder () const |
Returns kTRUE in case object contains browsable objects (like containers or lists of other objects). | |
R__ALWAYS_INLINE Bool_t | IsOnHeap () const |
R__ALWAYS_INLINE Bool_t | IsZombie () const |
void | MayNotUse (const char *method) const |
Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary). | |
virtual Bool_t | Notify () |
This method must be overridden to handle object notification. | |
void | Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const |
Use this method to declare a method obsolete. | |
void | operator delete (void *ptr) |
Operator delete. | |
void | operator delete[] (void *ptr) |
Operator delete []. | |
void * | operator new (size_t sz) |
void * | operator new (size_t sz, void *vp) |
void * | operator new[] (size_t sz) |
void * | operator new[] (size_t sz, void *vp) |
TObject & | operator= (const TObject &rhs) |
TObject assignment operator. | |
virtual void | Paint (Option_t *option="") |
This method must be overridden if a class wants to paint itself. | |
virtual void | Pop () |
Pop on object drawn in a pad to the top of the display list. | |
virtual Int_t | Read (const char *name) |
Read contents of object with specified name from the current directory. | |
virtual void | RecursiveRemove (TObject *obj) |
Recursively remove this object from a list. | |
void | ResetBit (UInt_t f) |
virtual void | SaveAs (const char *filename="", Option_t *option="") const |
Save this object in the file specified by filename. | |
virtual void | SavePrimitive (std::ostream &out, Option_t *option="") |
Save a primitive as a C++ statement(s) on output stream "out". | |
void | SetBit (UInt_t f) |
void | SetBit (UInt_t f, Bool_t set) |
Set or unset the user status bits as specified in f. | |
virtual void | SetDrawOption (Option_t *option="") |
Set drawing option for object. | |
virtual void | SetUniqueID (UInt_t uid) |
Set the unique object id. | |
virtual void | SysError (const char *method, const char *msgfmt,...) const |
Issue system error message. | |
R__ALWAYS_INLINE Bool_t | TestBit (UInt_t f) const |
Int_t | TestBits (UInt_t f) const |
virtual void | UseCurrentStyle () |
Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked. | |
virtual void | Warning (const char *method, const char *msgfmt,...) const |
Issue warning message. | |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) |
Write this object to the current directory. | |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const |
Write this object to the current directory. | |
Protected Attributes | |
const RooArgList * | fAllTestStatisticsData |
RooDataSet * | fAltDetailedOutput |
SamplingDistribution * | fAltDistr |
Double_t | fAlternatePValue |
Double_t | fAlternatePValueError |
Bool_t | fBackgroundIsAlt |
std::unique_ptr< RooDataSet > | fFitInfo |
RooDataSet * | fNullDetailedOutput |
SamplingDistribution * | fNullDistr |
Double_t | fNullPValue |
Double_t | fNullPValueError |
Bool_t | fPValueIsRightTail |
Double_t | fTestStatisticData |
Protected Attributes inherited from TNamed | |
TString | fName |
TString | fTitle |
Private Member Functions | |
void | UpdatePValue (const SamplingDistribution *distr, Double_t &pvalue, Double_t &perror, Bool_t pIsRightTail) |
updates the pvalue if sufficient data is available | |
Additional Inherited Members | |
Public Types inherited from TObject | |
enum | { kIsOnHeap = 0x01000000 , kNotDeleted = 0x02000000 , kZombie = 0x04000000 , kInconsistent = 0x08000000 , kBitMask = 0x00ffffff } |
enum | { kSingleKey = BIT(0) , kOverwrite = BIT(1) , kWriteDelete = BIT(2) } |
enum | EDeprecatedStatusBits { kObjInCanvas = BIT(3) } |
enum | EStatusBits { kCanDelete = BIT(0) , kMustCleanup = BIT(3) , kIsReferenced = BIT(4) , kHasUUID = BIT(5) , kCannotPick = BIT(6) , kNoContextMenu = BIT(8) , kInvalidObject = BIT(13) } |
Static Public Member Functions inherited from TObject | |
static Longptr_t | GetDtorOnly () |
Return destructor only flag. | |
static Bool_t | GetObjectStat () |
Get status of object stat flag. | |
static void | SetDtorOnly (void *obj) |
Set destructor only flag. | |
static void | SetObjectStat (Bool_t stat) |
Turn on/off tracking of objects in the TObjectTable. | |
Protected Types inherited from TObject | |
enum | { kOnlyPrepStep = BIT(3) } |
Protected Member Functions inherited from TObject | |
virtual void | DoError (int level, const char *location, const char *fmt, va_list va) const |
Interface to ErrorHandler (protected). | |
void | MakeZombie () |
#include <RooStats/HypoTestResult.h>
|
explicit |
HypoTestResult::HypoTestResult | ( | const HypoTestResult & | other | ) |
copy constructor
Definition at line 109 of file HypoTestResult.cxx.
constructor from name, null and alternate p values
Alternate constructor.
Definition at line 93 of file HypoTestResult.cxx.
|
virtual |
|
inlinevirtual |
Return p-value for alternate hypothesis.
Reimplemented in RooStats::HybridResult.
Definition at line 48 of file HypoTestResult.h.
|
virtual |
add values from another HypoTestResult
Add additional toy-MC experiments to the current results.
Use the data test statistics of the added object if it is not already set (otherwise, ignore the new one).
Definition at line 171 of file HypoTestResult.cxx.
|
inlinevirtual |
Convert NullPValue into a "confidence level".
Definition at line 51 of file HypoTestResult.h.
Double_t HypoTestResult::CLbError | ( | ) | const |
The error on the "confidence level" of the null hypothesis.
compute \(CL_{b}\) error \(CL_{b}\) = 1 - NullPValue() must use opposite condition that routine above
Definition at line 273 of file HypoTestResult.cxx.
|
inlinevirtual |
\(CL_{s}\) is simply \(CL_{s+b}/CL_{b}\) (not a method, but a quantity)
Definition at line 57 of file HypoTestResult.h.
Double_t HypoTestResult::CLsError | ( | ) | const |
The error on the ratio \(CL_{s+b}/CL_{b}\).
Returns an estimate of the error on \(CL_{s}\) through combination of the errors on \(CL_{b}\) and \(CL_{s+b}\):
\[ \sigma_{CL_s} = CL_s \sqrt{\left( \frac{\sigma_{CL_{s+b}}}{CL_{s+b}} \right)^2 + \left( \frac{\sigma_{CL_{b}}}{CL_{b}} \right)^2} \]
Definition at line 298 of file HypoTestResult.cxx.
|
inlinevirtual |
Convert AlternatePValue into a "confidence level".
Definition at line 54 of file HypoTestResult.h.
Double_t HypoTestResult::CLsplusbError | ( | ) | const |
The error on the "confidence level" of the alternative hypothesis.
Definition at line 279 of file HypoTestResult.cxx.
|
inline |
Definition at line 76 of file HypoTestResult.h.
|
inline |
Definition at line 73 of file HypoTestResult.h.
|
inline |
Definition at line 71 of file HypoTestResult.h.
Definition at line 91 of file HypoTestResult.h.
|
inline |
Definition at line 74 of file HypoTestResult.h.
|
inline |
Definition at line 72 of file HypoTestResult.h.
|
inline |
Definition at line 70 of file HypoTestResult.h.
Definition at line 88 of file HypoTestResult.h.
Definition at line 75 of file HypoTestResult.h.
Definition at line 257 of file HypoTestResult.cxx.
|
inlinevirtual |
Return p-value for null hypothesis.
Reimplemented in RooStats::HybridResult.
Definition at line 45 of file HypoTestResult.h.
Double_t HypoTestResult::NullPValueError | ( | ) | const |
The error on the Null p-value.
Definition at line 263 of file HypoTestResult.cxx.
HypoTestResult & HypoTestResult::operator= | ( | const HypoTestResult & | other | ) |
assignment operator
Definition at line 140 of file HypoTestResult.cxx.
Print out some information about the results Note: use Alt/Null labels for the hypotheses here as the Null might be the s+b hypothesis.
Definition at line 339 of file HypoTestResult.cxx.
void HypoTestResult::SetAllTestStatisticsData | ( | const RooArgList * | tsd | ) |
Definition at line 233 of file HypoTestResult.cxx.
|
inline |
Definition at line 81 of file HypoTestResult.h.
void HypoTestResult::SetAltDistribution | ( | SamplingDistribution * | alt | ) |
Definition at line 210 of file HypoTestResult.cxx.
Definition at line 90 of file HypoTestResult.h.
|
inline |
Definition at line 83 of file HypoTestResult.h.
|
inline |
Definition at line 82 of file HypoTestResult.h.
void HypoTestResult::SetNullDistribution | ( | SamplingDistribution * | null | ) |
Definition at line 217 of file HypoTestResult.cxx.
Definition at line 248 of file HypoTestResult.cxx.
Definition at line 224 of file HypoTestResult.cxx.
|
inlinevirtual |
familiar name for the Null p-value in terms of 1-sided Gaussian significance
Definition at line 68 of file HypoTestResult.h.
Double_t HypoTestResult::SignificanceError | ( | ) | const |
The error on the significance, computed from NullPValueError via error propagation.
Taylor expansion series approximation for standard deviation (error propagation)
Definition at line 286 of file HypoTestResult.cxx.
|
private |
updates the pvalue if sufficient data is available
Definition at line 316 of file HypoTestResult.cxx.
|
protected |
Definition at line 122 of file HypoTestResult.h.
|
protected |
Definition at line 126 of file HypoTestResult.h.
|
protected |
Definition at line 124 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 118 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 120 of file HypoTestResult.h.
|
protected |
Definition at line 129 of file HypoTestResult.h.
|
protected |
Definition at line 127 of file HypoTestResult.h.
|
protected |
Definition at line 125 of file HypoTestResult.h.
|
protected |
Definition at line 123 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 117 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 119 of file HypoTestResult.h.
|
protected |
Definition at line 128 of file HypoTestResult.h.
|
protected |
Definition at line 121 of file HypoTestResult.h.