{
"cells": [
{
"cell_type": "markdown",
"id": "c2d16606",
"metadata": {},
"source": [
"# rf610_visualerror\n",
"Likelihood and minimization: visualization of errors from a covariance matrix\n",
"\n",
"\n",
"\n",
"\n",
"**Author:** Wouter Verkerke \n",
"This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Tuesday, March 19, 2024 at 07:17 PM."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "1a802e62",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:08.905795Z",
"iopub.status.busy": "2024-03-19T19:17:08.905276Z",
"iopub.status.idle": "2024-03-19T19:17:08.961242Z",
"shell.execute_reply": "2024-03-19T19:17:08.957822Z"
}
},
"outputs": [],
"source": [
"%%cpp -d\n",
"#include \"RooRealVar.h\"\n",
"#include \"RooDataHist.h\"\n",
"#include \"RooGaussian.h\"\n",
"#include \"RooAddPdf.h\"\n",
"#include \"RooPlot.h\"\n",
"#include \"TCanvas.h\"\n",
"#include \"TAxis.h\"\n",
"#include \"TAxis.h\"\n",
"using namespace RooFit;"
]
},
{
"cell_type": "markdown",
"id": "f1f7bf7f",
"metadata": {},
"source": [
"Setup example fit\n",
"---------------------------------------"
]
},
{
"cell_type": "markdown",
"id": "4a42c781",
"metadata": {},
"source": [
"Create sum of two Gaussians pdf with factory"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "9c8a7b68",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:08.976334Z",
"iopub.status.busy": "2024-03-19T19:17:08.975933Z",
"iopub.status.idle": "2024-03-19T19:17:10.587800Z",
"shell.execute_reply": "2024-03-19T19:17:10.576531Z"
}
},
"outputs": [],
"source": [
"RooRealVar x(\"x\", \"x\", -10, 10);\n",
"\n",
"RooRealVar m(\"m\", \"m\", 0, -10, 10);\n",
"RooRealVar s(\"s\", \"s\", 2, 1, 50);\n",
"RooGaussian sig(\"sig\", \"sig\", x, m, s);\n",
"\n",
"RooRealVar m2(\"m2\", \"m2\", -1, -10, 10);\n",
"RooRealVar s2(\"s2\", \"s2\", 6, 1, 50);\n",
"RooGaussian bkg(\"bkg\", \"bkg\", x, m2, s2);\n",
"\n",
"RooRealVar fsig(\"fsig\", \"fsig\", 0.33, 0, 1);\n",
"RooAddPdf model(\"model\", \"model\", RooArgList(sig, bkg), fsig);"
]
},
{
"cell_type": "markdown",
"id": "cb7eae25",
"metadata": {},
"source": [
"Create binned dataset"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "c0c0b9fa",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:10.606959Z",
"iopub.status.busy": "2024-03-19T19:17:10.606568Z",
"iopub.status.idle": "2024-03-19T19:17:10.831708Z",
"shell.execute_reply": "2024-03-19T19:17:10.829059Z"
}
},
"outputs": [],
"source": [
"x.setBins(25);\n",
"std::unique_ptr d{model.generateBinned(x, 1000)};"
]
},
{
"cell_type": "markdown",
"id": "9c101731",
"metadata": {},
"source": [
"Perform fit and save fit result"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "c0c695ea",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:10.860697Z",
"iopub.status.busy": "2024-03-19T19:17:10.860318Z",
"iopub.status.idle": "2024-03-19T19:17:11.327404Z",
"shell.execute_reply": "2024-03-19T19:17:11.322255Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Fitting -- RooAbsPdf::fitTo(model) fixing normalization set for coefficient determination to observables in data\n",
"[#1] INFO:Fitting -- using CPU computation library compiled with -mavx2\n",
"[#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_model_genData) Summation contains a RooNLLVar, using its error level\n",
"[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: activating const optimization\n",
"[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization\n"
]
}
],
"source": [
"std::unique_ptr r{model.fitTo(*d, Save(), PrintLevel(-1))};"
]
},
{
"cell_type": "markdown",
"id": "81c2c0aa",
"metadata": {},
"source": [
"Visualize fit error\n",
"-------------------------------------"
]
},
{
"cell_type": "markdown",
"id": "c49a376b",
"metadata": {},
"source": [
"Make plot frame"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "ed2ee484",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:11.364091Z",
"iopub.status.busy": "2024-03-19T19:17:11.363720Z",
"iopub.status.idle": "2024-03-19T19:17:11.581230Z",
"shell.execute_reply": "2024-03-19T19:17:11.579621Z"
}
},
"outputs": [],
"source": [
"RooPlot *frame = x.frame(Bins(40), Title(\"P.d.f with visualized 1-sigma error band\"));\n",
"d->plotOn(frame);"
]
},
{
"cell_type": "markdown",
"id": "66c1d566",
"metadata": {},
"source": [
"Visualize 1-sigma error encoded in fit result 'r' as orange band using linear error propagation\n",
"This results in an error band that is by construction symmetric\n",
"\n",
"The linear error is calculated as\n",
"error(x) = Z* F_a(x) * Corr(a,a') F_a'(x)\n",
"\n",
"where F_a(x) = [ f(x,a+da) - f(x,a-da) ] / 2,\n",
"\n",
"with f(x) = the plotted curve\n",
"'da' = error taken from the fit result\n",
"Corr(a,a') = the correlation matrix from the fit result\n",
"Z = requested significance 'Z sigma band'\n",
"\n",
"The linear method is fast (required 2*N evaluations of the curve, where N is the number of parameters),\n",
"but may not be accurate in the presence of strong correlations (~>0.9) and at Z>2 due to linear and\n",
"Gaussian approximations made"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f2744bff",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:11.591442Z",
"iopub.status.busy": "2024-03-19T19:17:11.590945Z",
"iopub.status.idle": "2024-03-19T19:17:11.859877Z",
"shell.execute_reply": "2024-03-19T19:17:11.858604Z"
}
},
"outputs": [],
"source": [
"model.plotOn(frame, VisualizeError(*r, 1), FillColor(kOrange));"
]
},
{
"cell_type": "markdown",
"id": "c26fd3cd",
"metadata": {},
"source": [
"Calculate error using sampling method and visualize as dashed red line.\n",
"\n",
"In this method a number of curves is calculated with variations of the parameter values, as sampled\n",
"from a multi-variate Gaussian pdf that is constructed from the fit results covariance matrix.\n",
"The error(x) is determined by calculating a central interval that capture N% of the variations\n",
"for each value of x, where N% is controlled by Z (i.e. Z=1 gives N=68%). The number of sampling curves\n",
"is chosen to be such that at least 100 curves are expected to be outside the N% interval, and is minimally\n",
"100 (e.g. Z=1->Ncurve=356, Z=2->Ncurve=2156)) Intervals from the sampling method can be asymmetric,\n",
"and may perform better in the presence of strong correlations, but may take (much) longer to calculate"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "b4b0eb2c",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:11.870399Z",
"iopub.status.busy": "2024-03-19T19:17:11.870009Z",
"iopub.status.idle": "2024-03-19T19:17:12.304688Z",
"shell.execute_reply": "2024-03-19T19:17:12.303499Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Plotting -- RooAbsReal::plotOn(model) INFO: visualizing 1-sigma uncertainties in parameters (m,s,fsig,m2,s2) from fit result fitresult_model_genData using 315 samplings.\n"
]
}
],
"source": [
"model.plotOn(frame, VisualizeError(*r, 1, false), DrawOption(\"L\"), LineWidth(2), LineColor(kRed));"
]
},
{
"cell_type": "markdown",
"id": "45c9d17d",
"metadata": {},
"source": [
"Perform the same type of error visualization on the background component only.\n",
"The VisualizeError() option can generally applied to _any_ kind of plot (components, asymmetries, efficiencies\n",
"etc..)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "7ed4b2c5",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:12.328128Z",
"iopub.status.busy": "2024-03-19T19:17:12.327735Z",
"iopub.status.idle": "2024-03-19T19:17:12.703349Z",
"shell.execute_reply": "2024-03-19T19:17:12.701413Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n",
"[#1] INFO:Plotting -- RooAbsReal::plotOn(model) INFO: visualizing 1-sigma uncertainties in parameters (m,s,fsig,m2,s2) from fit result fitresult_model_genData using 315 samplings.\n"
]
}
],
"source": [
"model.plotOn(frame, VisualizeError(*r, 1), FillColor(kOrange), Components(\"bkg\"));\n",
"model.plotOn(frame, VisualizeError(*r, 1, false), DrawOption(\"L\"), LineWidth(2), LineColor(kRed), Components(\"bkg\"),\n",
" LineStyle(kDashed));"
]
},
{
"cell_type": "markdown",
"id": "5c281646",
"metadata": {},
"source": [
"Overlay central value"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "71d9621a",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:12.718753Z",
"iopub.status.busy": "2024-03-19T19:17:12.718352Z",
"iopub.status.idle": "2024-03-19T19:17:12.936870Z",
"shell.execute_reply": "2024-03-19T19:17:12.935845Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n"
]
}
],
"source": [
"model.plotOn(frame);\n",
"model.plotOn(frame, Components(\"bkg\"), LineStyle(kDashed));\n",
"d->plotOn(frame);\n",
"frame->SetMinimum(0);"
]
},
{
"cell_type": "markdown",
"id": "fb6664e8",
"metadata": {},
"source": [
"Visualize partial fit error\n",
"------------------------------------------------------"
]
},
{
"cell_type": "markdown",
"id": "54abcad4",
"metadata": {},
"source": [
"Make plot frame"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "94cf56c4",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:12.943598Z",
"iopub.status.busy": "2024-03-19T19:17:12.943138Z",
"iopub.status.idle": "2024-03-19T19:17:13.169696Z",
"shell.execute_reply": "2024-03-19T19:17:13.157787Z"
}
},
"outputs": [],
"source": [
"RooPlot *frame2 = x.frame(Bins(40), Title(\"Visualization of 2-sigma partial error from (m,m2)\"));"
]
},
{
"cell_type": "markdown",
"id": "37e850a5",
"metadata": {},
"source": [
"Visualize partial error. For partial error visualization the covariance matrix is first reduced as follows\n",
"___ -1\n",
"Vred = V22 = V11 - V12 * V22 * V21\n",
"\n",
"Where V11,V12,V21,V22 represent a block decomposition of the covariance matrix into observables that\n",
"are propagated (labeled by index '1') and that are not propagated (labeled by index '2'), and V22bar\n",
"is the Shur complement of V22, calculated as shown above\n",
"\n",
"(Note that Vred is _not_ a simple sub-matrix of V)"
]
},
{
"cell_type": "markdown",
"id": "50adf866",
"metadata": {},
"source": [
"Propagate partial error due to shape parameters (m,m2) using linear and sampling method"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "d5fef829",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:13.189800Z",
"iopub.status.busy": "2024-03-19T19:17:13.189414Z",
"iopub.status.idle": "2024-03-19T19:17:13.556232Z",
"shell.execute_reply": "2024-03-19T19:17:13.555235Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n"
]
}
],
"source": [
"model.plotOn(frame2, VisualizeError(*r, RooArgSet(m, m2), 2), FillColor(kCyan));\n",
"model.plotOn(frame2, Components(\"bkg\"), VisualizeError(*r, RooArgSet(m, m2), 2), FillColor(kCyan));\n",
"\n",
"model.plotOn(frame2);\n",
"model.plotOn(frame2, Components(\"bkg\"), LineStyle(kDashed));\n",
"frame2->SetMinimum(0);"
]
},
{
"cell_type": "markdown",
"id": "26ca6ae2",
"metadata": {},
"source": [
"Make plot frame"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "09f031b0",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:13.570180Z",
"iopub.status.busy": "2024-03-19T19:17:13.569800Z",
"iopub.status.idle": "2024-03-19T19:17:13.792923Z",
"shell.execute_reply": "2024-03-19T19:17:13.791999Z"
}
},
"outputs": [],
"source": [
"RooPlot *frame3 = x.frame(Bins(40), Title(\"Visualization of 2-sigma partial error from (s,s2)\"));"
]
},
{
"cell_type": "markdown",
"id": "b2917b51",
"metadata": {},
"source": [
"Propagate partial error due to yield parameter using linear and sampling method"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "d7a4c324",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:13.797868Z",
"iopub.status.busy": "2024-03-19T19:17:13.797429Z",
"iopub.status.idle": "2024-03-19T19:17:14.046915Z",
"shell.execute_reply": "2024-03-19T19:17:14.029807Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n"
]
}
],
"source": [
"model.plotOn(frame3, VisualizeError(*r, RooArgSet(s, s2), 2), FillColor(kGreen));\n",
"model.plotOn(frame3, Components(\"bkg\"), VisualizeError(*r, RooArgSet(s, s2), 2), FillColor(kGreen));\n",
"\n",
"model.plotOn(frame3);\n",
"model.plotOn(frame3, Components(\"bkg\"), LineStyle(kDashed));\n",
"frame3->SetMinimum(0);"
]
},
{
"cell_type": "markdown",
"id": "273cb7e0",
"metadata": {},
"source": [
"Make plot frame"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "41ffc31f",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:14.059030Z",
"iopub.status.busy": "2024-03-19T19:17:14.058622Z",
"iopub.status.idle": "2024-03-19T19:17:14.272782Z",
"shell.execute_reply": "2024-03-19T19:17:14.270998Z"
}
},
"outputs": [],
"source": [
"RooPlot *frame4 = x.frame(Bins(40), Title(\"Visualization of 2-sigma partial error from fsig\"));"
]
},
{
"cell_type": "markdown",
"id": "b29c15a2",
"metadata": {},
"source": [
"Propagate partial error due to yield parameter using linear and sampling method"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "8eda3edd",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:14.280212Z",
"iopub.status.busy": "2024-03-19T19:17:14.279855Z",
"iopub.status.idle": "2024-03-19T19:17:14.528622Z",
"shell.execute_reply": "2024-03-19T19:17:14.522480Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) directly selected PDF components: (bkg)\n",
"[#1] INFO:Plotting -- RooAbsPdf::plotOn(model) indirectly selected PDF components: ()\n"
]
}
],
"source": [
"model.plotOn(frame4, VisualizeError(*r, RooArgSet(fsig), 2), FillColor(kMagenta));\n",
"model.plotOn(frame4, Components(\"bkg\"), VisualizeError(*r, RooArgSet(fsig), 2), FillColor(kMagenta));\n",
"\n",
"model.plotOn(frame4);\n",
"model.plotOn(frame4, Components(\"bkg\"), LineStyle(kDashed));\n",
"frame4->SetMinimum(0);\n",
"\n",
"TCanvas *c = new TCanvas(\"rf610_visualerror\", \"rf610_visualerror\", 800, 800);\n",
"c->Divide(2, 2);\n",
"c->cd(1);\n",
"gPad->SetLeftMargin(0.15);\n",
"frame->GetYaxis()->SetTitleOffset(1.4);\n",
"frame->Draw();\n",
"c->cd(2);\n",
"gPad->SetLeftMargin(0.15);\n",
"frame2->GetYaxis()->SetTitleOffset(1.6);\n",
"frame2->Draw();\n",
"c->cd(3);\n",
"gPad->SetLeftMargin(0.15);\n",
"frame3->GetYaxis()->SetTitleOffset(1.6);\n",
"frame3->Draw();\n",
"c->cd(4);\n",
"gPad->SetLeftMargin(0.15);\n",
"frame4->GetYaxis()->SetTitleOffset(1.6);\n",
"frame4->Draw();"
]
},
{
"cell_type": "markdown",
"id": "9b4a5be1",
"metadata": {},
"source": [
"Draw all canvases "
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "3810d96c",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:14.550005Z",
"iopub.status.busy": "2024-03-19T19:17:14.549614Z",
"iopub.status.idle": "2024-03-19T19:17:15.198195Z",
"shell.execute_reply": "2024-03-19T19:17:15.197255Z"
}
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"
\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"%jsroot on\n",
"gROOT->GetListOfCanvases()->Draw()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "ROOT C++",
"language": "c++",
"name": "root"
},
"language_info": {
"codemirror_mode": "text/x-c++src",
"file_extension": ".C",
"mimetype": " text/x-c++src",
"name": "c++"
}
},
"nbformat": 4,
"nbformat_minor": 5
}