The Levenberg-Marquardt plugin is used to fit an SBML model’s parameters to experimental data.

The plugin has numerous properties to allow the user full control over the internal fitting engine, as well as access to generated fitted data after a minimization session. In addition, various statistical properties, such as standardized residuals, Q-Q data, ChiSquare and reduced ChiSquare are made accessible to the user. The resulting parameter values also come with estimated confidence limits.

The current implementation is based on the lmfit C library by Joachim Wuttke¹ .

1.2 Plugin Properties

Available properties in the Levenberg-Marquardt plugin are listed in the table below.

The following properties are used internally by the fitting engine. They are pre-set with default values. Depending on the minimization problem at hand, they may need to be tweaked.

Table 1.1: Levenberg-Marquardt plugin properties

Property Name	Data Type	Default Value	Description


SBML	string	N/A	SBML document as a string. Model to be used in the fitting.
ExperimentalData	telluriumData	N/A	Input data.
FittedData	telluriumData	N/A	Output data.
InputParameterList	listOfProperties	N/A	Parameters to fit.
OutputParameterList	listOfProperties	N/A	List of fitted parameters.
ExperimentalDataSelectionList	stringList	N/A	Species selection list for experimental data.
FittedDataSelectionList	stringList	N/A	Selection list for model data.
Norm	double	N/A	Norm of fitting. An estimate of goodness of fit.
Norms	telluriumData	N/A	The norm is calculated throughout a fitting session. Each Norm value is stored in the Norms (read-only) property.
ConfidenceLimits	listOfProperties	N/A	Confidence limits for each fitted parameter. The confidence limits are calculated at a 95% confidence level.
Hessian	matrix	N/A	Hessian matrix. The Hessian is calculated using approximation at a found parameter minimum.
CovarianceMatrix	matrix	N/A	Covariance matrix. Calculated as the inverse of the Hessian.
Residuals	telluriumData	N/A	Residuals data.
StandardizedResiduals	telluriumData	N/A	Standardized Residuals.
NormalProbabilityOfResiduals	telluriumData	N/A	Normal Probability of Residuals.
ChiSquare	double	N/A	The ChiSquare at the minimum.
ReducedChiSquare	double	N/A	The Reduced ChiSquare at the minimum.
StatusMessage	string	N/A	Message from the internal fitting engine, communicating the status of the obtained fit.
NrOfIter	int	N/A	Number of iterations.










ftol	double	machine dep.	Relative error desired in the sum of squares.
xtol	double	machine dep.	Relative error between last two approximations.
gtol	double	machine dep.	Orthogonality desired between fvec and its derivs.
epsilon	double	machine dep.	Step used to calculate the Jacobian.
stepbound	double	100.0	Initial bound to steps in the outer loop.
patience	double	100	Used for setting maximum number of iterations, calculated as patience*(nr_of_parameters +1).

1.3 The execute(bool inThread) function

The execute() function will start the Levenberg-Marquardt algorithm. Depending on the problem at hand, the algorithm may run for a long time.

The execute(bool inThread) method supports a boolean argument indicating if the execution of the plugin work will be done in a thread, or not. Threading is fully implemented in the Levenberg-Marquardt plugin.

1.4 Plugin Events

The Levenberg-Marquardt plugin are using all of a plugins available plugin events, i.e. the PluginStarted, PluginProgress and the PluginFinished events.

The available data variables for each event are internally treated as pass through variables, so any data, for any of the events, assigned prior to the plugin’s execute function (in the assignOn() family of functions), can be retrieved unmodified in the corresponding event function.

1.5 Python example

1from teplugins import *
2
3# Load Plugins
4chiPlugin       = Plugin("tel_chisquare")
5lm              = Plugin("tel_levenberg_marquardt")
6modelPlugin     = Plugin("tel_test_model")
7addNoisePlugin  = Plugin("tel_add_noise")
8
9try:
10    #========== EVENT FUNCTION SETUP ===========================
11    def myEvent(dummy): #We are not capturing any data from the plugin, so just pass a dummy
12        print ’Iteration, Norm = ’ + ‘lm.getProperty("NrOfIter")‘  + ’, ’ + ‘lm.getProperty("Norm")‘
13
14    #Setup progress event function
15    progressEvent =  NotifyEventEx(myEvent)
16    assignOnProgressEvent(lm.plugin, progressEvent)
17    #============================================================
18
19    #Create model data, with and without noise using the test_model plugin
20    modelPlugin.execute()
21
22    #Setup lmfit properties.
23    lm.SBML             = modelPlugin.Model
24    lm.ExperimentalData = modelPlugin.TestDataWithNoise
25
26    # Add the parameters that we’re going to fit and an initial ’start’ value
27    lm.setProperty("InputParameterList", ["k1", .3])
28    lm.setProperty("FittedDataSelectionList", "[S1] [S2]")
29    lm.setProperty("ExperimentalDataSelectionList", "[S1] [S2]")
30
31    # Start minimization
32    lm.execute()
33
34    print ’Minimization finished. \n==== Result ====’
35    print ’Fit engine status: ’ + ‘lm.getProperty(’StatusMessage’)‘
36
37    print ’Hessian Matrix’
38    print lm.getProperty("Hessian")
39
40    print ’Covariance  Matrix’
41    print lm.getProperty("CovarianceMatrix")
42
43    print ’ChiSquare = ’            + ‘lm.getProperty("ChiSquare")‘
44    print ’Reduced ChiSquare = ’    + ‘lm.getProperty("ReducedChiSquare")‘
45
46    #This is a list of parameters
47    parameters = tpc.getPluginProperty (lm.plugin, "OutputParameterList")
48    confLimits = tpc.getPluginProperty (lm.plugin, "ConfidenceLimits")
49
50    #Iterate trough list of parameters and confidence limits
51    para  = getFirstProperty(parameters)
52    limit = getFirstProperty(confLimits)
53    while para and limit:
54        print getPropertyName(para) + ’ = ’ + ‘getPropertyValue(para)‘ + ’ +/- ’ + ‘getPropertyValue(limit)‘
55        para  = getNextProperty(parameters)
56        limit = getNextProperty(confLimits)
57
58
59    # Get the fitted and residual data
60    fittedData = lm.getProperty ("FittedData").toNumpy
61    residuals  = lm.getProperty ("Residuals").toNumpy
62
63    # Get the experimental data as a numpy array
64    experimentalData = modelPlugin.TestDataWithNoise.toNumpy
65
66    telplugins.plot(fittedData         [:,[0,1]], "blue", "-",    "",    "S1 Fitted")
67    telplugins.plot(fittedData         [:,[0,2]], "blue", "-",    "",    "S2 Fitted")
68    telplugins.plot(residuals          [:,[0,1]], "blue", "None", "x",   "S1 Residual")
69    telplugins.plot(residuals          [:,[0,2]], "red",  "None", "x",   "S2 Residual")
70    telplugins.plot(experimentalData   [:,[0,1]], "red",  "",     "*",   "S1 Data")
71    telplugins.plot(experimentalData   [:,[0,2]], "blue", "",     "*",   "S2 Data")
72    telplugins.plt.show()
73
74except Exception as e:
75    print ’Problem.. ’ + ‘e‘

Listing 1.1: Minimization example.

Event	Arguments	Purpose and argument types


PluginStarted	void, void	Signals to application that the plugin has started. Both parameters are pass through parameters and are unused internally by the plugin.

PluginProgress	void, void	Communicates progress of fitting. Both parameters are pass through parameters and are unused internally by the plugin.

PluginFinished	void, void	Signals to application that execution of the plugin has finished. Both parameters are pass through parameters and are unused internally by the plugin.

Parameter Minimization using the Levenberg-Marquardt algorithm

1.1 Introduction

1.2 Plugin Properties

1.3 The execute(bool inThread) function

1.4 Plugin Events

1.5 Python example