Hypothesis Test Calculator based on the asymptotic formulae for the profile likelihood ratio.

It performs hypothesis tests using the asymptotic formula for the profile likelihood, and uses the Asimov data set to compute expected significances or limits.

See G. Cowan, K. Cranmer, E. Gross and O. Vitells: Asymptotic formulae for likelihood- based tests of new physics. Eur. Phys. J., C71:1–19, 2011. It provides methods to perform hypothesis tests using the likelihood function, and computes the \(p\)-values for the null and the alternate hypothesis using the asymptotic formulae for the profile likelihood ratio described in the given paper.

The calculator provides methods to produce the Asimov dataset, i.e. a dataset generated where the observed values are equal to the expected ones. The Asimov data set is then used to compute the observed asymptotic \(p\)-value for the alternate hypothesis and the asymptotic expected \(p\)-values.

The asymptotic formulae are valid only for one POI (parameter of interest). So the calculator works only for one-dimensional (one POI) models. If more than one POI exists, only the first one is used.

The calculator can generate Asimov datasets from two kinds of PDFs:

"Counting" distributions: RooPoisson, RooGaussian, or products of RooPoissons.
Extended, i.e. number of events can be read off from extended likelihood term.

Definition at line 27 of file AsymptoticCalculator.h.

Public Member Functions
	AsymptoticCalculator (RooAbsData &data, const ModelConfig &altModel, const ModelConfig &nullModel, bool nominalAsimov=false)
	constructor for asymptotic calculator from Data set and ModelConfig
const ModelConfig *	GetAlternateModel (void) const
const RooArgSet &	GetBestFitParams () const
	return best fit value for all parameters
const RooArgSet &	GetBestFitPoi () const
	return snapshot of the best fit parameter
const RooAbsData *	GetData (void) const
virtual const RooArgSet *	GetFitInfo () const
HypoTestResult *	GetHypoTest () const override
	re-implement HypoTest computation using the asymptotic
const RooRealVar *	GetMuHat () const
	return best fit parameter (firs of poi)
const ModelConfig *	GetNullModel (void) const
TestStatSampler *	GetTestStatSampler (void) const
	Returns instance of TestStatSampler.
bool	Initialize () const
	initialize the calculator by performing a global fit and make the Asimov data set
TClass *	IsA () const override
bool	IsOneSidedDiscovery () const
bool	IsTwoSided () const
void	SetAlternateModel (const ModelConfig &altModel) override
	Set the model for the alternate hypothesis.
virtual void	SetCommonModel (const ModelConfig &model)
	Set a common model for both the null and alternate, add to the workspace if not already there.
void	SetData (RooAbsData &data) override
	Set the DataSet.
void	SetNullModel (const ModelConfig &nullModel) override
	re-implementation of setters since they are needed to re-initialize the calculator
void	SetOneSided (bool on)
	set test statistic for one sided (upper limits)
void	SetOneSidedDiscovery (bool on)
	set the test statistics for one-sided discovery
void	SetQTilde (bool on)
	set using of qtilde, by default is controlled if RoORealVar is limited or not
void	SetTwoSided ()
	set the test statistics for two sided (in case of upper limits for discovery does not make really sense)
void	Streamer (TBuffer &) override
void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)
void	UseSameAltToys ()
	Set this for re-using always the same toys for alternate hypothesis in case of calls at different null parameter points This is useful to get more stable bands when running the HypoTest inversion.

Static Public Member Functions
static TClass *	Class ()
static const char *	Class_Name ()
static constexpr Version_t	Class_Version ()
static const char *	DeclFileName ()
static RooAbsData *	GenerateAsimovData (const RooAbsPdf &pdf, const RooArgSet &observables)
	generate the asimov data for the observables (not the global ones) need to deal with the case of a sim pdf
static double	GetExpectedPValues (double pnull, double palt, double nsigma, bool usecls, bool oneSided=true)
	function given the null and the alt p value - return the expected one given the N - sigma value
static RooAbsData *	MakeAsimovData (const ModelConfig &model, const RooArgSet &allParamValues, RooArgSet &globObs)
	Make a nominal Asimov data set from a model.
static RooAbsData *	MakeAsimovData (RooAbsData &data, const ModelConfig &model, const RooArgSet &poiValues, RooArgSet &globObs, const RooArgSet *genPoiValues=nullptr)
	Make Asimov data.
static void	SetPrintLevel (int level)
	set print level (static function)

Protected Member Functions
virtual int	CheckHook (void) const
virtual void	PostHook () const
virtual int	PreAltHook (RooArgSet *, double) const
virtual void	PreHook () const
virtual int	PreNullHook (RooArgSet *, double) const

Protected Attributes
const ModelConfig *	fAltModel
unsigned int	fAltToysSeed
const RooAbsData *	fData
TestStatSampler *	fDefaultSampler
TestStatistic *	fDefaultTestStat
const ModelConfig *	fNullModel
TestStatSampler *	fTestStatSampler

Private Member Functions
SamplingDistribution *	GenerateSamplingDistribution (ModelConfig thisModel, double obsTestStat, RooAbsPdf impDens=nullptr, const RooArgSet *impSnapshot=nullptr) const
void	SetAdaptiveLimits (double obsTestStat, bool forNull) const
void	SetupSampler (const ModelConfig &model) const
	common setup for both models

Private Attributes
RooAbsData *	fAsimovData
	asimov data set
RooArgSet	fAsimovGlobObs
	snapshot of Asimov global observables
RooArgSet	fBestFitParams
	snapshot of all best fitted Parameter values
RooArgSet	fBestFitPoi
	snapshot of best fitted POI values
bool	fIsInitialized
	! flag to check if calculator is initialized
double	fNLLAsimov
double	fNLLObs
bool	fNominalAsimov
	make Asimov at nominal parameter values
bool	fOneSided
	for one sided PL test statistic (upper limits)
bool	fOneSidedDiscovery
	for one sided PL test statistic (for discovery)
int	fUseQTilde
	flag to indicate if using qtilde or not (-1 (default based on RooRealVar)), 0 false, 1 (true)

#include <RooStats/AsymptoticCalculator.h>

Inheritance diagram for RooStats::AsymptoticCalculator:

Constructor & Destructor Documentation

◆ AsymptoticCalculator()

AsymptoticCalculator::AsymptoticCalculator	(	RooAbsData &	data,
		const ModelConfig &	altModel,
		const ModelConfig &	nullModel,
		bool	nominalAsimov = false )

constructor for asymptotic calculator from Data set and ModelConfig

Definition at line 107 of file AsymptoticCalculator.cxx.

Member Function Documentation

◆ CheckHook()

virtual int RooStats::HypoTestCalculatorGeneric::CheckHook ( void ) const

inlineprotectedvirtualinherited

Reimplemented in RooStats::HybridCalculator.

Definition at line 73 of file HypoTestCalculatorGeneric.h.

◆ Class()

TClass * RooStats::AsymptoticCalculator::Class ( )

static

Returns: TClass describing this class

◆ Class_Name()

const char * RooStats::AsymptoticCalculator::Class_Name ( )

static

Returns: Name of this class

◆ Class_Version()

constexpr Version_t RooStats::AsymptoticCalculator::Class_Version ( )

inlinestaticconstexpr

Returns: Version of this class

Definition at line 112 of file AsymptoticCalculator.h.

◆ DeclFileName()

const char * RooStats::AsymptoticCalculator::DeclFileName ( )

inlinestatic

Returns: Name of the file containing the class declaration

Definition at line 112 of file AsymptoticCalculator.h.

◆ GenerateAsimovData()

RooAbsData * AsymptoticCalculator::GenerateAsimovData	(	const RooAbsPdf &	pdf,
		const RooArgSet &	observables )

static

generate the asimov data for the observables (not the global ones) need to deal with the case of a sim pdf

Parameters

pdf	a RooAbsPdf. Can be also a RooSimultaneous, or a RooProPdf of a RooSimultaneous

Definition at line 1165 of file AsymptoticCalculator.cxx.

◆ GenerateSamplingDistribution()

SamplingDistribution * RooStats::HypoTestCalculatorGeneric::GenerateSamplingDistribution	(	ModelConfig *	thisModel,
		double	obsTestStat,
		RooAbsPdf *	impDens = nullptr,
		const RooArgSet *	impSnapshot = nullptr ) const

privateinherited

◆ GetAlternateModel()

const ModelConfig * RooStats::HypoTestCalculatorGeneric::GetAlternateModel ( void ) const

inlineinherited

Definition at line 57 of file HypoTestCalculatorGeneric.h.

◆ GetBestFitParams()

const RooArgSet & RooStats::AsymptoticCalculator::GetBestFitParams ( ) const

inline

return best fit value for all parameters

Definition at line 94 of file AsymptoticCalculator.h.

◆ GetBestFitPoi()

const RooArgSet & RooStats::AsymptoticCalculator::GetBestFitPoi ( ) const

inline

return snapshot of the best fit parameter

Definition at line 90 of file AsymptoticCalculator.h.

◆ GetData()

const RooAbsData * RooStats::HypoTestCalculatorGeneric::GetData ( void ) const

inlineinherited

Definition at line 52 of file HypoTestCalculatorGeneric.h.

◆ GetExpectedPValues()

double AsymptoticCalculator::GetExpectedPValues	(	double	pnull,
		double	palt,
		double	nsigma,
		bool	usecls,
		bool	oneSided = true )

static

function given the null and the alt p value - return the expected one given the N - sigma value

Definition at line 813 of file AsymptoticCalculator.cxx.

◆ GetFitInfo()

virtual const RooArgSet * RooStats::HypoTestCalculatorGeneric::GetFitInfo ( ) const

inlinevirtualinherited

Reimplemented in RooStats::FrequentistCalculator.

Definition at line 54 of file HypoTestCalculatorGeneric.h.

◆ GetHypoTest()

HypoTestResult * AsymptoticCalculator::GetHypoTest ( ) const

overridevirtual

re-implement HypoTest computation using the asymptotic

It performs an hypothesis tests using the likelihood function and computes the p values for the null and the alternate using the asymptotic formulae for the profile likelihood ratio.

See G. Cowan, K. Cranmer, E. Gross and O. Vitells. Asymptotic formulae for likelihood- based tests of new physics. Eur. Phys. J., C71:1–19, 2011. The formulae are valid only for one POI. If more than one POI exists consider as POI only the first one

Implements RooStats::HypoTestCalculator.

Definition at line 480 of file AsymptoticCalculator.cxx.

◆ GetMuHat()

const RooRealVar * RooStats::AsymptoticCalculator::GetMuHat ( ) const

inline

return best fit parameter (firs of poi)

Definition at line 92 of file AsymptoticCalculator.h.

◆ GetNullModel()

const ModelConfig * RooStats::HypoTestCalculatorGeneric::GetNullModel ( void ) const

inlineinherited

Definition at line 53 of file HypoTestCalculatorGeneric.h.

◆ GetTestStatSampler()

TestStatSampler * RooStats::HypoTestCalculatorGeneric::GetTestStatSampler ( void ) const

inlineinherited

Returns instance of TestStatSampler.

Use to change properties of TestStatSampler, e.g. GetTestStatSampler.SetTestSize(double size);

Definition at line 63 of file HypoTestCalculatorGeneric.h.

◆ Initialize()

bool AsymptoticCalculator::Initialize ( ) const

initialize the calculator by performing a global fit and make the Asimov data set

Initialize the calculator The initialization will perform a global fit of the model to the data and build an Asimov data set.

It will then also fit the model to the Asimov data set to find the likelihood value of the Asimov data set nominalAsimov is an option for using Asimov data set obtained using nominal nuisance parameter values By default the nuisance parameters are fitted to the data NOTE: If a fit has been done before, one for speeding up could set all the initial parameters to the fit value and in addition set the null snapshot to the best fit

Definition at line 145 of file AsymptoticCalculator.cxx.

◆ IsA()

TClass * RooStats::AsymptoticCalculator::IsA ( ) const

inlineoverridevirtual

Returns: TClass describing current object

Reimplemented from RooStats::HypoTestCalculator.

Definition at line 112 of file AsymptoticCalculator.h.

◆ IsOneSidedDiscovery()

bool RooStats::AsymptoticCalculator::IsOneSidedDiscovery ( ) const

inline

Definition at line 83 of file AsymptoticCalculator.h.

◆ IsTwoSided()

bool RooStats::AsymptoticCalculator::IsTwoSided ( ) const

inline

Definition at line 82 of file AsymptoticCalculator.h.

◆ MakeAsimovData() [1/2]

RooAbsData * AsymptoticCalculator::MakeAsimovData	(	const ModelConfig &	model,
		const RooArgSet &	allParamValues,
		RooArgSet &	asimovGlobObs )

static

Make a nominal Asimov data set from a model.

Parameters

	model	ModelConfig that contains the model pdf and the model parameters
	allParamValues	The parameters of the model will be set to the values given in this set
[out]	asimovGlobObs	Global observables set to values satisfying the constraints

Returns: Asimov data set. The user takes ownership.

The parameter values (including the nuisance parameter) can result from a fit to data or be at the nominal values.

Definition at line 1370 of file AsymptoticCalculator.cxx.

◆ MakeAsimovData() [2/2]

RooAbsData * AsymptoticCalculator::MakeAsimovData	(	RooAbsData &	realData,
		const ModelConfig &	model,
		const RooArgSet &	paramValues,
		RooArgSet &	asimovGlobObs,
		const RooArgSet *	genPoiValues = nullptr )

static

Make Asimov data.

Make the Asimov data from the ModelConfig and list of poi.

Parameters

	realData	Real data
	model	Model config defining the pdf and the parameters
	paramValues	The snapshot of POI and parameters used for finding the best nuisance parameter values (conditioned at these values)
[out]	asimovGlobObs	Global observables set to values satisfying the constraints
	genPoiValues	Optional. A different set of POI values used for generating. By default the same POI are used for generating and for finding the nuisance parameters given an observed data set, a model and a snapshot of the poi.

Returns: The asimov data set. The user takes ownership.

Definition at line 1260 of file AsymptoticCalculator.cxx.

◆ PostHook()

virtual void RooStats::HypoTestCalculatorGeneric::PostHook ( ) const

inlineprotectedvirtualinherited

Reimplemented in RooStats::FrequentistCalculator.

Definition at line 77 of file HypoTestCalculatorGeneric.h.

◆ PreAltHook()

virtual int RooStats::HypoTestCalculatorGeneric::PreAltHook	(	RooArgSet *	,
		double	) const

inlineprotectedvirtualinherited

Reimplemented in RooStats::FrequentistCalculator, and RooStats::HybridCalculator.

Definition at line 75 of file HypoTestCalculatorGeneric.h.

◆ PreHook()

virtual void RooStats::HypoTestCalculatorGeneric::PreHook ( ) const

inlineprotectedvirtualinherited

Reimplemented in RooStats::FrequentistCalculator.

Definition at line 76 of file HypoTestCalculatorGeneric.h.

◆ PreNullHook()

virtual int RooStats::HypoTestCalculatorGeneric::PreNullHook	(	RooArgSet *	,
		double	) const

inlineprotectedvirtualinherited

Reimplemented in RooStats::FrequentistCalculator, and RooStats::HybridCalculator.

Definition at line 74 of file HypoTestCalculatorGeneric.h.

◆ SetAdaptiveLimits()

void RooStats::HypoTestCalculatorGeneric::SetAdaptiveLimits	(	double	obsTestStat,
		bool	forNull ) const

privateinherited

◆ SetAlternateModel()

void RooStats::AsymptoticCalculator::SetAlternateModel ( const ModelConfig & model )

inlineoverridevirtual

Set the model for the alternate hypothesis.

Implements RooStats::HypoTestCalculator.

Definition at line 72 of file AsymptoticCalculator.h.

◆ SetCommonModel()

virtual void RooStats::HypoTestCalculator::SetCommonModel ( const ModelConfig & model )

inlinevirtualinherited

Set a common model for both the null and alternate, add to the workspace if not already there.

Definition at line 70 of file HypoTestCalculator.h.

◆ SetData()

void RooStats::AsymptoticCalculator::SetData ( RooAbsData & data )

inlineoverridevirtual

Set the DataSet.

Implements RooStats::HypoTestCalculator.

Definition at line 76 of file AsymptoticCalculator.h.

◆ SetNullModel()

void RooStats::AsymptoticCalculator::SetNullModel ( const ModelConfig & nullModel )

inlineoverridevirtual

re-implementation of setters since they are needed to re-initialize the calculator

Implements RooStats::HypoTestCalculator.

Definition at line 68 of file AsymptoticCalculator.h.

◆ SetOneSided()

void RooStats::AsymptoticCalculator::SetOneSided ( bool on )

inline

set test statistic for one sided (upper limits)

Definition at line 58 of file AsymptoticCalculator.h.

◆ SetOneSidedDiscovery()

void RooStats::AsymptoticCalculator::SetOneSidedDiscovery ( bool on )

inline

set the test statistics for one-sided discovery

Definition at line 65 of file AsymptoticCalculator.h.

◆ SetPrintLevel()

void AsymptoticCalculator::SetPrintLevel ( int level )

static

set print level (static function)

0 minimal,
1 normal,
2 debug

Definition at line 100 of file AsymptoticCalculator.cxx.

◆ SetQTilde()

void RooStats::AsymptoticCalculator::SetQTilde ( bool on )

inline

set using of qtilde, by default is controlled if RoORealVar is limited or not

Definition at line 87 of file AsymptoticCalculator.h.

◆ SetTwoSided()

void RooStats::AsymptoticCalculator::SetTwoSided ( )

inline

set the test statistics for two sided (in case of upper limits for discovery does not make really sense)

Definition at line 62 of file AsymptoticCalculator.h.

◆ SetupSampler()

void HypoTestCalculatorGeneric::SetupSampler ( const ModelConfig & model ) const

privateinherited

common setup for both models

Definition at line 88 of file HypoTestCalculatorGeneric.cxx.

◆ Streamer()

void RooStats::AsymptoticCalculator::Streamer ( TBuffer & )

overridevirtual

Reimplemented from RooStats::HypoTestCalculator.

◆ StreamerNVirtual()

void RooStats::AsymptoticCalculator::StreamerNVirtual ( TBuffer & ClassDef_StreamerNVirtual_b )

inline

Definition at line 112 of file AsymptoticCalculator.h.

◆ UseSameAltToys()

void HypoTestCalculatorGeneric::UseSameAltToys ( )

inherited

Set this for re-using always the same toys for alternate hypothesis in case of calls at different null parameter points This is useful to get more stable bands when running the HypoTest inversion.

to re-use same toys for alternate hypothesis

Definition at line 276 of file HypoTestCalculatorGeneric.cxx.

Member Data Documentation

◆ fAltModel

const ModelConfig* RooStats::HypoTestCalculatorGeneric::fAltModel

protectedinherited

Definition at line 80 of file HypoTestCalculatorGeneric.h.

◆ fAltToysSeed

unsigned int RooStats::HypoTestCalculatorGeneric::fAltToysSeed

protectedinherited

Definition at line 87 of file HypoTestCalculatorGeneric.h.

◆ fAsimovData

RooAbsData* RooStats::AsymptoticCalculator::fAsimovData

mutableprivate

asimov data set

Definition at line 107 of file AsymptoticCalculator.h.

◆ fAsimovGlobObs

RooArgSet RooStats::AsymptoticCalculator::fAsimovGlobObs

mutableprivate

snapshot of Asimov global observables

Definition at line 108 of file AsymptoticCalculator.h.

◆ fBestFitParams

RooArgSet RooStats::AsymptoticCalculator::fBestFitParams

mutableprivate

snapshot of all best fitted Parameter values

Definition at line 110 of file AsymptoticCalculator.h.

◆ fBestFitPoi

RooArgSet RooStats::AsymptoticCalculator::fBestFitPoi

mutableprivate

snapshot of best fitted POI values

Definition at line 109 of file AsymptoticCalculator.h.

◆ fData

const RooAbsData* RooStats::HypoTestCalculatorGeneric::fData

protectedinherited

Definition at line 82 of file HypoTestCalculatorGeneric.h.

◆ fDefaultSampler

TestStatSampler* RooStats::HypoTestCalculatorGeneric::fDefaultSampler

protectedinherited

Definition at line 84 of file HypoTestCalculatorGeneric.h.

◆ fDefaultTestStat

TestStatistic* RooStats::HypoTestCalculatorGeneric::fDefaultTestStat

protectedinherited

Definition at line 85 of file HypoTestCalculatorGeneric.h.

◆ fIsInitialized

bool RooStats::AsymptoticCalculator::fIsInitialized

mutableprivate

! flag to check if calculator is initialized

Definition at line 102 of file AsymptoticCalculator.h.

◆ fNLLAsimov

double RooStats::AsymptoticCalculator::fNLLAsimov

mutableprivate

Definition at line 105 of file AsymptoticCalculator.h.

◆ fNLLObs

double RooStats::AsymptoticCalculator::fNLLObs

mutableprivate

Definition at line 104 of file AsymptoticCalculator.h.

◆ fNominalAsimov

bool RooStats::AsymptoticCalculator::fNominalAsimov

private

make Asimov at nominal parameter values

Definition at line 101 of file AsymptoticCalculator.h.

◆ fNullModel

const ModelConfig* RooStats::HypoTestCalculatorGeneric::fNullModel

protectedinherited

Definition at line 81 of file HypoTestCalculatorGeneric.h.

◆ fOneSided

bool RooStats::AsymptoticCalculator::fOneSided

private

for one sided PL test statistic (upper limits)

Definition at line 99 of file AsymptoticCalculator.h.

◆ fOneSidedDiscovery

bool RooStats::AsymptoticCalculator::fOneSidedDiscovery

mutableprivate

for one sided PL test statistic (for discovery)

Definition at line 100 of file AsymptoticCalculator.h.

◆ fTestStatSampler

TestStatSampler* RooStats::HypoTestCalculatorGeneric::fTestStatSampler

protectedinherited

Definition at line 83 of file HypoTestCalculatorGeneric.h.

◆ fUseQTilde

int RooStats::AsymptoticCalculator::fUseQTilde

mutableprivate

flag to indicate if using qtilde or not (-1 (default based on RooRealVar)), 0 false, 1 (true)

Definition at line 103 of file AsymptoticCalculator.h.

The documentation for this class was generated from the following files:

roofit/roostats/inc/RooStats/AsymptoticCalculator.h
roofit/roostats/src/AsymptoticCalculator.cxx

Public Member Functions

Static Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Private Attributes

Constructor & Destructor Documentation

◆ AsymptoticCalculator()

Member Function Documentation

◆ CheckHook()

◆ Class()

◆ Class_Name()

◆ Class_Version()

◆ DeclFileName()

◆ GenerateAsimovData()

◆ GenerateSamplingDistribution()

◆ GetAlternateModel()

◆ GetBestFitParams()

◆ GetBestFitPoi()

◆ GetData()

◆ GetExpectedPValues()

◆ GetFitInfo()

◆ GetHypoTest()

◆ GetMuHat()

◆ GetNullModel()

◆ GetTestStatSampler()

◆ Initialize()

◆ IsA()

◆ IsOneSidedDiscovery()

◆ IsTwoSided()

◆ MakeAsimovData() [1/2]

◆ MakeAsimovData() [2/2]

◆ PostHook()

◆ PreAltHook()

◆ PreHook()

◆ PreNullHook()

◆ SetAdaptiveLimits()

◆ SetAlternateModel()

◆ SetCommonModel()

◆ SetData()

◆ SetNullModel()

◆ SetOneSided()

◆ SetOneSidedDiscovery()

◆ SetPrintLevel()

◆ SetQTilde()

◆ SetTwoSided()

◆ SetupSampler()

◆ Streamer()

◆ StreamerNVirtual()

◆ UseSameAltToys()

Member Data Documentation

◆ fAltModel

◆ fAltToysSeed

◆ fAsimovData

◆ fAsimovGlobObs

◆ fBestFitParams

◆ fBestFitPoi

◆ fData

◆ fDefaultSampler

◆ fDefaultTestStat

◆ fIsInitialized

◆ fNLLAsimov

◆ fNLLObs

◆ fNominalAsimov

◆ fNullModel

◆ fOneSided

◆ fOneSidedDiscovery

◆ fTestStatSampler

◆ fUseQTilde