{
"cells": [
{
"cell_type": "markdown",
"id": "841656ec",
"metadata": {},
"source": [
"# rf606_nllerrorhandling\n",
"Likelihood and minimization: understanding and customizing error handling in likelihood evaluations\n",
"\n",
"\n",
"\n",
"\n",
"**Author:** Wouter Verkerke \n",
"This notebook tutorial was automatically generated with ROOTBOOK-izer from the macro found in the ROOT repository on Tuesday, March 19, 2024 at 07:16 PM."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "73caf085",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:00.275249Z",
"iopub.status.busy": "2024-03-19T19:17:00.274824Z",
"iopub.status.idle": "2024-03-19T19:17:00.364426Z",
"shell.execute_reply": "2024-03-19T19:17:00.350706Z"
}
},
"outputs": [],
"source": [
"%%cpp -d\n",
"#include \"RooRealVar.h\"\n",
"#include \"RooDataSet.h\"\n",
"#include \"RooArgusBG.h\"\n",
"#include \"TCanvas.h\"\n",
"#include \"TAxis.h\"\n",
"#include \"RooPlot.h\"\n",
"using namespace RooFit;"
]
},
{
"cell_type": "markdown",
"id": "814c74d9",
"metadata": {},
"source": [
"Create model and dataset\n",
"----------------------------------------------"
]
},
{
"cell_type": "markdown",
"id": "e135746e",
"metadata": {},
"source": [
"Observable"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "0030cc9f",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:00.390803Z",
"iopub.status.busy": "2024-03-19T19:17:00.390406Z",
"iopub.status.idle": "2024-03-19T19:17:01.396156Z",
"shell.execute_reply": "2024-03-19T19:17:01.395071Z"
}
},
"outputs": [],
"source": [
"RooRealVar m(\"m\", \"m\", 5.20, 5.30);"
]
},
{
"cell_type": "markdown",
"id": "ef6f1b79",
"metadata": {},
"source": [
"Parameters"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "e8146f19",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:01.454018Z",
"iopub.status.busy": "2024-03-19T19:17:01.453622Z",
"iopub.status.idle": "2024-03-19T19:17:01.674909Z",
"shell.execute_reply": "2024-03-19T19:17:01.671068Z"
}
},
"outputs": [],
"source": [
"RooRealVar m0(\"m0\", \"m0\", 5.291, 5.20, 5.30);\n",
"RooRealVar k(\"k\", \"k\", -30, -50, -10);"
]
},
{
"cell_type": "markdown",
"id": "fb02e5ce",
"metadata": {},
"source": [
"Pdf"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "897dddec",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:01.690723Z",
"iopub.status.busy": "2024-03-19T19:17:01.690347Z",
"iopub.status.idle": "2024-03-19T19:17:02.139828Z",
"shell.execute_reply": "2024-03-19T19:17:02.119961Z"
}
},
"outputs": [],
"source": [
"RooArgusBG argus(\"argus\", \"argus\", m, m0, k);"
]
},
{
"cell_type": "markdown",
"id": "b44c54b9",
"metadata": {},
"source": [
"Sample 1000 events in m from argus"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "da700eca",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:02.165408Z",
"iopub.status.busy": "2024-03-19T19:17:02.165028Z",
"iopub.status.idle": "2024-03-19T19:17:02.694010Z",
"shell.execute_reply": "2024-03-19T19:17:02.691155Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"input_line_51:2:2: warning: 'data' shadows a declaration with the same name in the 'std' namespace; use '::data' to reference this declaration\n",
" std::unique_ptr data{argus.generate(m, 1000)};\n",
" ^\n"
]
}
],
"source": [
"std::unique_ptr data{argus.generate(m, 1000)};"
]
},
{
"cell_type": "markdown",
"id": "241c1238",
"metadata": {},
"source": [
"Plot model and data\n",
"--------------------------------------"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "6dab0fd5",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:02.724028Z",
"iopub.status.busy": "2024-03-19T19:17:02.723589Z",
"iopub.status.idle": "2024-03-19T19:17:02.988666Z",
"shell.execute_reply": "2024-03-19T19:17:02.951680Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"input_line_52:3:1: error: reference to 'data' is ambiguous\n",
"data->plotOn(frame1);\n",
"^\n",
"input_line_51:2:30: note: candidate found by name lookup is 'data'\n",
" std::unique_ptr data{argus.generate(m, 1000)};\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:318:5: note: candidate found by name lookup is 'std::data'\n",
" data(initializer_list<_Tp> __il) noexcept\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:289:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:299:5: note: candidate found by name lookup is 'std::data'\n",
" data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:309:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Tp (&__array)[_Nm]) noexcept\n",
" ^\n"
]
}
],
"source": [
"RooPlot *frame1 = m.frame(Bins(40), Title(\"Argus model and data\"));\n",
"data->plotOn(frame1);\n",
"argus.plotOn(frame1);"
]
},
{
"cell_type": "markdown",
"id": "640e5486",
"metadata": {},
"source": [
"Fit model to data\n",
"---------------------------------"
]
},
{
"cell_type": "markdown",
"id": "eb4dc9bb",
"metadata": {},
"source": [
"The ARGUS background shape has a sharp kinematic cutoff at m=m0\n",
"and is prone to evaluation errors if the cutoff parameter m0\n",
"is floated: when the pdf cutoff value is lower than that in data\n",
"events with m>m0 will have zero probability"
]
},
{
"cell_type": "markdown",
"id": "4fc005e6",
"metadata": {},
"source": [
"Perform unbinned ML fit. Print detailed error messages for up to\n",
"10 events per likelihood evaluation. The default error handling strategy\n",
"is to return a very high value of the likelihood to MINUIT if errors occur,\n",
"which will force MINUIT to retreat from the problematic area"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "95a5026f",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:02.997111Z",
"iopub.status.busy": "2024-03-19T19:17:02.996494Z",
"iopub.status.idle": "2024-03-19T19:17:03.280411Z",
"shell.execute_reply": "2024-03-19T19:17:03.272697Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"input_line_53:2:15: error: reference to 'data' is ambiguous\n",
" argus.fitTo(*data, PrintEvalErrors(10));\n",
" ^\n",
"input_line_51:2:30: note: candidate found by name lookup is 'data'\n",
" std::unique_ptr data{argus.generate(m, 1000)};\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:318:5: note: candidate found by name lookup is 'std::data'\n",
" data(initializer_list<_Tp> __il) noexcept\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:289:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:299:5: note: candidate found by name lookup is 'std::data'\n",
" data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:309:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Tp (&__array)[_Nm]) noexcept\n",
" ^\n"
]
}
],
"source": [
"argus.fitTo(*data, PrintEvalErrors(10));"
]
},
{
"cell_type": "markdown",
"id": "40e080cf",
"metadata": {},
"source": [
"Perform another fit. In this configuration only the number of errors per\n",
"likelihood evaluation is shown, if it is greater than zero. The\n",
"EvalErrorWall(false) arguments disables the default error handling strategy\n",
"and will cause the actual (problematic) value of the likelihood to be passed\n",
"to MINUIT.\n",
"\n",
"NB: Use of this option is NOT recommended as default strategy as broken -log(L) values\n",
"can often be lower than 'good' ones because offending events are removed.\n",
"This may effectively create a false minimum in problem areas. This is clearly\n",
"illustrated in the second plot"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "001553e8",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:03.313457Z",
"iopub.status.busy": "2024-03-19T19:17:03.312964Z",
"iopub.status.idle": "2024-03-19T19:17:03.595426Z",
"shell.execute_reply": "2024-03-19T19:17:03.576786Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"input_line_54:3:14: error: reference to 'data' is ambiguous\n",
"argus.fitTo(*data, PrintEvalErrors(0), EvalErrorWall(false));\n",
" ^\n",
"input_line_51:2:30: note: candidate found by name lookup is 'data'\n",
" std::unique_ptr data{argus.generate(m, 1000)};\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:318:5: note: candidate found by name lookup is 'std::data'\n",
" data(initializer_list<_Tp> __il) noexcept\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:289:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:299:5: note: candidate found by name lookup is 'std::data'\n",
" data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:309:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Tp (&__array)[_Nm]) noexcept\n",
" ^\n"
]
}
],
"source": [
"m0.setError(0.1);\n",
"argus.fitTo(*data, PrintEvalErrors(0), EvalErrorWall(false));"
]
},
{
"cell_type": "markdown",
"id": "4b714900",
"metadata": {},
"source": [
"Plot likelihood as function of m0\n",
"------------------------------------------------------------------"
]
},
{
"cell_type": "markdown",
"id": "a049ed9f",
"metadata": {},
"source": [
"Construct likelihood function of model and data"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "8c3c8425",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:03.605952Z",
"iopub.status.busy": "2024-03-19T19:17:03.605572Z",
"iopub.status.idle": "2024-03-19T19:17:03.852253Z",
"shell.execute_reply": "2024-03-19T19:17:03.849386Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"input_line_55:2:51: error: reference to 'data' is ambiguous\n",
" std::unique_ptr nll{argus.createNLL(*data)};\n",
" ^\n",
"input_line_51:2:30: note: candidate found by name lookup is 'data'\n",
" std::unique_ptr data{argus.generate(m, 1000)};\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:318:5: note: candidate found by name lookup is 'std::data'\n",
" data(initializer_list<_Tp> __il) noexcept\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:289:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:299:5: note: candidate found by name lookup is 'std::data'\n",
" data(const _Container& __cont) noexcept(noexcept(__cont.data()))\n",
" ^\n",
"/usr/include/c++/9/bits/range_access.h:309:5: note: candidate found by name lookup is 'std::data'\n",
" data(_Tp (&__array)[_Nm]) noexcept\n",
" ^\n"
]
}
],
"source": [
"std::unique_ptr nll{argus.createNLL(*data)};"
]
},
{
"cell_type": "markdown",
"id": "00e46743",
"metadata": {},
"source": [
"Plot likelihood in m0 in range that includes problematic values\n",
"In this configuration no messages are printed for likelihood evaluation errors,\n",
"but if an likelihood value evaluates with error, the corresponding value\n",
"on the curve will be set to the value given in EvalErrorValue()."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "8d33be9f",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:03.857364Z",
"iopub.status.busy": "2024-03-19T19:17:03.857000Z",
"iopub.status.idle": "2024-03-19T19:17:04.109880Z",
"shell.execute_reply": "2024-03-19T19:17:04.099620Z"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"input_line_56:3:103: error: cannot take the address of an rvalue of type 'EColor'\n",
"nll->plotOn(frame2, PrintEvalErrors(-1), ShiftToZero(), EvalErrorValue(nll->getVal() + 10), LineColor(kRed));\n",
" ^~~~\n",
"Error while creating dynamic expression for:\n",
" nll->plotOn(frame2, PrintEvalErrors(-1), ShiftToZero(), EvalErrorValue(nll->getVal() + 10), LineColor(kRed))\n"
]
}
],
"source": [
"RooPlot *frame2 = m0.frame(Range(5.288, 5.293), Title(\"-log(L) scan vs m0, problematic regions masked\"));\n",
"nll->plotOn(frame2, PrintEvalErrors(-1), ShiftToZero(), EvalErrorValue(nll->getVal() + 10), LineColor(kRed));\n",
"frame2->SetMaximum(15);\n",
"frame2->SetMinimum(0);\n",
"\n",
"TCanvas *c = new TCanvas(\"rf606_nllerrorhandling\", \"rf606_nllerrorhandling\", 1200, 400);\n",
"c->Divide(2);\n",
"c->cd(1);\n",
"gPad->SetLeftMargin(0.15);\n",
"frame1->GetYaxis()->SetTitleOffset(1.4);\n",
"frame1->Draw();\n",
"c->cd(2);\n",
"gPad->SetLeftMargin(0.15);\n",
"frame2->GetYaxis()->SetTitleOffset(1.4);\n",
"frame2->Draw();"
]
},
{
"cell_type": "markdown",
"id": "4fc68e18",
"metadata": {},
"source": [
"Draw all canvases "
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "79043b71",
"metadata": {
"collapsed": false,
"execution": {
"iopub.execute_input": "2024-03-19T19:17:04.121763Z",
"iopub.status.busy": "2024-03-19T19:17:04.121378Z",
"iopub.status.idle": "2024-03-19T19:17:04.441481Z",
"shell.execute_reply": "2024-03-19T19:17:04.434070Z"
}
},
"outputs": [],
"source": [
"%jsroot on\n",
"gROOT->GetListOfCanvases()->Draw()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "ROOT C++",
"language": "c++",
"name": "root"
},
"language_info": {
"codemirror_mode": "text/x-c++src",
"file_extension": ".C",
"mimetype": " text/x-c++src",
"name": "c++"
}
},
"nbformat": 4,
"nbformat_minor": 5
}