Algorithms - Revision history

127.0.0.1: /* Solving */

2022-11-04T00:35:30Z

Solving

← Older revision		Revision as of 00:35, 4 November 2022
Line 60:		Line 60:
	Following the approach of Harville(1999)<ref name="Harville1999" /> {{lmt}} can sample for fixed factors the diagonal elements of the inverse of the mixed model coefficient matrix, for random factors the diagonal blocks of the inverse of the coefficient matrix where the block size is determined by the dimension of the related $$\Sigma$$ matrix. The blocks are the prediction error co-variance matrices of the factor levels of correlated sub-factors. When sampling prediction error variances {{lmt}} can run many Gibbs chains in parallel allowing to exploit multi-core hardware architecture. However, it is recommended to specify not more chains than the number of available '''real''' cores excluding hyper-threading technology.		Following the approach of Harville(1999)<ref name="Harville1999" /> {{lmt}} can sample for fixed factors the diagonal elements of the inverse of the mixed model coefficient matrix, for random factors the diagonal blocks of the inverse of the coefficient matrix where the block size is determined by the dimension of the related $$\Sigma$$ matrix. The blocks are the prediction error co-variance matrices of the factor levels of correlated sub-factors. When sampling prediction error variances {{lmt}} can run many Gibbs chains in parallel allowing to exploit multi-core hardware architecture. However, it is recommended to specify not more chains than the number of available '''real''' cores excluding hyper-threading technology.
	===Solving===		===Solving===
	{{lmt}} can obtain elements of the inverse of the coefficient matrix via solving the mixed model equations. This method is currently only supported for the diagonal prediction error co-variance blocks of random factors, where the block size is determined by the dimension of the related $$\Sigma$$ matrix. For this algorithm {{lmt}} can utilize either the [[#Iterative solver]] or the [[#Direct solver]].		{{lmt}} can obtain elements of the inverse of the coefficient matrix via solving the mixed model equations. This method is currently only supported for the diagonal prediction error co-variance blocks of random factors, where the block size is determined by the dimension of the related $$\Sigma$$ matrix. For this algorithm {{lmt}} can utilize either the [[#Iterative solver\|iterative solver]] or the [[#Direct solver\|direct solver]].

	==Iterative inbreeding==		==Iterative inbreeding==

127.0.0.1: /* Average information (AI)-REML */

2022-11-04T00:34:41Z

Average information (AI)-REML

← Older revision		Revision as of 00:34, 4 November 2022
Line 34:		Line 34:
	* $$log_e\left(\sqrt{\|\|g_{i}\|\|}\right)$$ where $$g$$ is the gradient vector and $$i$$ is the iteration counter.		* $$log_e\left(\sqrt{\|\|g_{i}\|\|}\right)$$ where $$g$$ is the gradient vector and $$i$$ is the iteration counter.

	=====REML Iteration mechanism=====		=====AI-REML Iteration mechanism=====

	For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$\zeta_i=\zeta_{i-1}+\alpha(\Omega_i+I\beta)^{-1}\xi_i$$, where $$\zeta_i$$ and $$\zeta_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$\Omega$$ is the AI matrix, $$\xi$$ is the Jacobian and $$\beta$$ is an arbitrary real number $$\geq$$0.		For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$\zeta_i=\zeta_{i-1}+\alpha(\Omega_i+I\beta)^{-1}\xi_i$$, where $$\zeta_i$$ and $$\zeta_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$\Omega$$ is the AI matrix, $$\xi$$ is the Jacobian and $$\beta$$ is an arbitrary real number $$\geq$$0.

127.0.0.1: /* REML Iteration mechanism */

2022-11-04T00:34:11Z

REML Iteration mechanism

← Older revision		Revision as of 00:34, 4 November 2022
Line 36:		Line 36:
	=====REML Iteration mechanism=====		=====REML Iteration mechanism=====

	For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$~~y_i~~=y_{i-1}+\alpha(~~A_i~~+I\beta)^{-1}~~j_i~~$$, where $$~~y_i~~$$ and $$y_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$A$$ is the AI matrix, $$j$$ is the Jacobian and $$\beta$$ is an arbitrary real number $$\geq$$0.		For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$\zeta_i=\zeta_{i-1}+\alpha(\Omega_i+I\beta)^{-1}\xi_i$$, where $$\zeta_i$$ and $$\zeta_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$\Omega$$ is the AI matrix, $$\xi$$ is the Jacobian and $$\beta$$ is an arbitrary real number $$\geq$$0.

	Once $$~~A_i~~$$ and $$~~j_i~~$$ are derived {{lmt}} will calculate $$~~y_i~~$$ using $$\alpha=1$$ and $$\beta=0$$. If $$~~y_i~~$$ is not valid(i.e. the $$\Sigma$$ matrices are not positive definite), it will use the Levenberg-Marquardt algorithm to find a valid $$~~y_i~~$$ by setting $$\beta$$ to an ever increasing number. {{lmt}} will try this for 10000 iterations. If $$~~y_i~~$$ is still not valid {{lmt}} will return to $$A_{i-1}$$ and $$j_{i-1}$$ and set $$\alpha=\alpha*0.5$$.		Once $$\Omega_i$$ and $$\xi_i$$ are derived {{lmt}} will calculate $$\zeta_i$$ using $$\alpha=1$$ and $$\beta=0$$. If $$\zeta_i$$ is not valid(i.e. the $$\Sigma$$ matrices are not positive definite), it will use the Levenberg-Marquardt algorithm to find a valid $$\zeta_i$$ by setting $$\beta$$ to an ever increasing number. {{lmt}} will try this for 10000 iterations. If $$\zeta_i$$ is still not valid {{lmt}} will return to $$\Omega_{i-1}$$ and $$\xi_{i-1}$$ and set $$\alpha=\alpha*0.5$$.

	This can lead to a situation where {{lmt}} down-scales $$\alpha$$ to near zero and the convergence criterion ~~appear~~ to ~~have converged~~. It is ~~there~~ crucial to check the iteration output. True convergence is only achieved if $$\alpha=1$$ and $$\beta=0$$.		This can lead to a situation where {{lmt}} down-scales $$\alpha$$ to near zero and the convergence criterion appears to report convergence. It is therefore crucial to check the iteration output. True convergence is only achieved if $$\alpha=1$$ and $$\beta=0$$.

	=====AI-REML-C=====		=====AI-REML-C=====

127.0.0.1: /* REML Iteration mechanism */

2022-11-04T00:30:11Z

REML Iteration mechanism

← Older revision		Revision as of 00:30, 4 November 2022
Line 36:		Line 36:
	=====REML Iteration mechanism=====		=====REML Iteration mechanism=====

	For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$y_i=y_{i-1}+\alpha(A_i+I\beta)^{-1}j_i$$, where $$y_i$$ and $$y_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$A$$ is the AI matrix, $$j$$ is the Jacobian and $$\beta$$ is an arbitrary ~~positive~~ number.		For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$y_i=y_{i-1}+\alpha(A_i+I\beta)^{-1}j_i$$, where $$y_i$$ and $$y_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$A$$ is the AI matrix, $$j$$ is the Jacobian and $$\beta$$ is an arbitrary real number $$\geq$$0.

	Once $$A_i$$ and $$j_i$$ are derived {{lmt}} will calculate $$y_i$$ using $$\alpha=1$$ and $$\beta=0$$. If $$y_i$$ is not valid(i.e. the $$\Sigma$$ matrices are not positive definite), it will use the Levenberg-Marquardt algorithm to find a valid $$y_i$$ by setting $$\beta$$ to an ever increasing number. {{lmt}} will try this for 10000 iterations. If $$y_i$$ is still not valid {{lmt}} will return to $$A_{i-1}$$ and $$j_{i-1}$$ and set $$\alpha=\alpha*0.5$$.		Once $$A_i$$ and $$j_i$$ are derived {{lmt}} will calculate $$y_i$$ using $$\alpha=1$$ and $$\beta=0$$. If $$y_i$$ is not valid(i.e. the $$\Sigma$$ matrices are not positive definite), it will use the Levenberg-Marquardt algorithm to find a valid $$y_i$$ by setting $$\beta$$ to an ever increasing number. {{lmt}} will try this for 10000 iterations. If $$y_i$$ is still not valid {{lmt}} will return to $$A_{i-1}$$ and $$j_{i-1}$$ and set $$\alpha=\alpha*0.5$$.

127.0.0.1: /* Average information (AI)-REML */

2022-11-03T23:42:13Z

Average information (AI)-REML

← Older revision		Revision as of 23:42, 3 November 2022
Line 34:		Line 34:
	* $$log_e\left(\sqrt{\|\|g_{i}\|\|}\right)$$ where $$g$$ is the gradient vector and $$i$$ is the iteration counter.		* $$log_e\left(\sqrt{\|\|g_{i}\|\|}\right)$$ where $$g$$ is the gradient vector and $$i$$ is the iteration counter.

	~~=====AI-REML-C=====~~		=====REML Iteration mechanism=====
	~~{{lmt}} supports AI-REML-C, which relies on the construction and factorization of the mixed-model equations system coefficient matrix C.~~

	~~Note that ssSNPBLUP and ssGTBLUP models are not supported. Further, it is not advisable to use airemlc for ssGBLUP models with several correlated genetic effects.~~

	======REML Iteration mechanism======

	For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$y_i=y_{i-1}+\alpha(A_i+I\beta)^{-1}j_i$$, where $$y_i$$ and $$y_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$A$$ is the AI matrix, $$j$$ is the Jacobian and $$\beta$$ is an arbitrary positive number.		For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$y_i=y_{i-1}+\alpha(A_i+I\beta)^{-1}j_i$$, where $$y_i$$ and $$y_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$A$$ is the AI matrix, $$j$$ is the Jacobian and $$\beta$$ is an arbitrary positive number.
Line 46:		Line 41:

	This can lead to a situation where {{lmt}} down-scales $$\alpha$$ to near zero and the convergence criterion appear to have converged. It is there crucial to check the iteration output. True convergence is only achieved if $$\alpha=1$$ and $$\beta=0$$.		This can lead to a situation where {{lmt}} down-scales $$\alpha$$ to near zero and the convergence criterion appear to have converged. It is there crucial to check the iteration output. True convergence is only achieved if $$\alpha=1$$ and $$\beta=0$$.

			=====AI-REML-C=====
			{{lmt}} supports AI-REML-C, which relies on the construction and factorization of the mixed-model equations system coefficient matrix C.

			Note that ssSNPBLUP and ssGTBLUP models are not supported. Further, it is not advisable to use airemlc for ssGBLUP models with several correlated genetic effects.

	===Fixation of Sigma matrix elements===		===Fixation of Sigma matrix elements===

127.0.0.1: /* REML Iteration mechanism */

2022-11-03T23:29:42Z

REML Iteration mechanism

← Older revision		Revision as of 23:29, 3 November 2022
Line 41:		Line 41:
	======REML Iteration mechanism======		======REML Iteration mechanism======

	For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm~~](#~~Levenberg–Marquardt algorithm) and ordinary step length down scaling.		For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg–Marquardt algorithm] and ordinary step length down scaling, which can be described as $$y_i=y_{i-1}+\alpha(A_i+I\beta)^{-1}j_i$$, where $$y_i$$ and $$y_{i-1}$$ are parameter vector is round $$i$$ and $$i-1$$, respectively, $$\alpha$$ is the step length, $$A$$ is the AI matrix, $$j$$ is the Jacobian and $$\beta$$ is an arbitrary positive number.

			Once $$A_i$$ and $$j_i$$ are derived {{lmt}} will calculate $$y_i$$ using $$\alpha=1$$ and $$\beta=0$$. If $$y_i$$ is not valid(i.e. the $$\Sigma$$ matrices are not positive definite), it will use the Levenberg-Marquardt algorithm to find a valid $$y_i$$ by setting $$\beta$$ to an ever increasing number. {{lmt}} will try this for 10000 iterations. If $$y_i$$ is still not valid {{lmt}} will return to $$A_{i-1}$$ and $$j_{i-1}$$ and set $$\alpha=\alpha*0.5$$.

			This can lead to a situation where {{lmt}} down-scales $$\alpha$$ to near zero and the convergence criterion appear to have converged. It is there crucial to check the iteration output. True convergence is only achieved if $$\alpha=1$$ and $$\beta=0$$.

	===Fixation of Sigma matrix elements===		===Fixation of Sigma matrix elements===

127.0.0.1: /* AI-REML-C */

2022-11-03T23:08:49Z

AI-REML-C

← Older revision		Revision as of 23:08, 3 November 2022
Line 37:		Line 37:
	{{lmt}} supports AI-REML-C, which relies on the construction and factorization of the mixed-model equations system coefficient matrix C.		{{lmt}} supports AI-REML-C, which relies on the construction and factorization of the mixed-model equations system coefficient matrix C.

	Note that ssSNPBLUP and ssGTBLUP models are not supported. Further, it is not advisable to use airemlc for ssGBLUP models.		Note that ssSNPBLUP and ssGTBLUP models are not supported. Further, it is not advisable to use airemlc for ssGBLUP models with several correlated genetic effects.

			======REML Iteration mechanism======

			For finding the next parameter vector {{lmt}} use a mixture of the [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm](#Levenberg–Marquardt algorithm) and ordinary step length down scaling.

	===Fixation of Sigma matrix elements===		===Fixation of Sigma matrix elements===

127.0.0.1: /* Fixation of $$\Sigma$$ matrix elements */

2022-06-07T04:40:03Z

Fixation of $$\Sigma$$ matrix elements

← Older revision		Revision as of 04:40, 7 June 2022
Line 39:		Line 39:
	Note that ssSNPBLUP and ssGTBLUP models are not supported. Further, it is not advisable to use airemlc for ssGBLUP models.		Note that ssSNPBLUP and ssGTBLUP models are not supported. Further, it is not advisable to use airemlc for ssGBLUP models.

	===Fixation of ~~$$\~~Sigma$$ matrix elements===		===Fixation of Sigma matrix elements===
	Elements of $$\Sigma$$ matrices can be exempted from re-estimation in two ways:		Elements of $$\Sigma$$ matrices can be exempted from re-estimation in two ways:
	#providing a boolean [[Parameter_file_elements#<sigma>\|mask matrix]] $$B$$ where elements set to "T" are related to elements in $$\Sigma$$ which should be regarded as fixed, or by		#providing a boolean [[Parameter_file_elements#<sigma>\|mask matrix]] $$B$$ where elements set to "T" are related to elements in $$\Sigma$$ which should be regarded as fixed, or by

127.0.0.1: /* Average information (AI)-REML */

2022-06-07T04:39:28Z

Average information (AI)-REML

← Older revision		Revision as of 04:39, 7 June 2022
Line 30:		Line 30:
	{{lmt}} provides three different AI-REML convergence criterions:		{{lmt}} provides three different AI-REML convergence criterions:

	* the relative change of the log-likelihood calculated as $$log_e\left(\sqrt{\frac{\|\|(l_{i}-l_{i-1})\|\|'}{\|\|l_{i-1}\|\|}}\right)$$ where $$l$$ is the log-likelihood and $$i$$ is the iteration counter.		* the relative change of the log-likelihood calculated as $$log_e\left(\sqrt{\frac{\|\|(l_{i}-l_{i-1})\|\|}{\|\|l_{i-1}\|\|}}\right)$$ where $$l$$ is the log-likelihood and $$i$$ is the iteration counter.
	* $$log_e\left(\sqrt{\frac{\|\|(p_{i}-p_{i-1})\|\|'}{\|\|p_{i-1}\|\|}}\right)$$ where $$p$$ is the parameter vector and $$i$$ is the iteration counter.		* $$log_e\left(\sqrt{\frac{\|\|(p_{i}-p_{i-1})\|\|}{\|\|p_{i-1}\|\|}}\right)$$ where $$p$$ is the parameter vector and $$i$$ is the iteration counter.
	* $$log_e\left(\sqrt{\|\|g_{i}\|\|}\right)$$ where $$g$$ is the gradient vector and $$i$$ is the iteration counter.		* $$log_e\left(\sqrt{\|\|g_{i}\|\|}\right)$$ where $$g$$ is the gradient vector and $$i$$ is the iteration counter.

127.0.0.1: /* MC-EM-REML */

2022-06-07T04:39:07Z

MC-EM-REML

← Older revision		Revision as of 04:39, 7 June 2022
Line 22:		Line 22:
	{{lmt}} provides a monte-carlo expectation-maximisation REML algorithms which uses the preconditioned gradient solver for solving the mixed model equations and a blocked Gibbs sampler to sample the necessary traces<ref name="Harville2004" />. Note that ssSNPBLUP and ssGTBLUP models are not supported.		{{lmt}} provides a monte-carlo expectation-maximisation REML algorithms which uses the preconditioned gradient solver for solving the mixed model equations and a blocked Gibbs sampler to sample the necessary traces<ref name="Harville2004" />. Note that ssSNPBLUP and ssGTBLUP models are not supported.

	The MC-EM-REML convergence criterion is $$log_e\left(\sqrt{\frac{\|\|(p_{i}-p_{i-1})\|\|'}{\|\|p_{i-1}\|\|}}\right)$$ where $$p$$ is the parameter vector and $$i$$ is the iteration counter.		The MC-EM-REML convergence criterion is $$log_e\left(\sqrt{\frac{\|\|(p_{i}-p_{i-1})\|\|}{\|\|p_{i-1}\|\|}}\right)$$ where $$p$$ is the parameter vector and $$i$$ is the iteration counter.

	====Average information (AI)-REML====		====Average information (AI)-REML====