Driverfile ========== .. _driverfile: .. code-block:: Driverfile The interface to DMU is based on a driver file in ASCII format. The information is organized in sections defined by keywords. Keywords marked with * are mandatory. .. _comment: $COMMENT ------- .. code-block:: Comment Specify that the following lines are comments to be put on page 1 of the listing file. $COMMENT Followed by up to 10 lines with up to 80 characters. .. _analyses: $ANALYSES -------- .. code-block:: Analyses Specify type of analyses and method to use. $ANALYSE task method scaling test_prt where: task = 1 -> REML estimation if (co)variances components using DMUAI. 2 -> RJMC. 11 -> BLUE AND BLUP using DMU4. 12 -> BLUE AND BLUP using DMU5. method = method to use. For task = 1 (REML) method can be: Sparse computation 1: AI, but combining AI and EM if an update goes outside the parameter space (the default). 2: EM based on an algorithm by Robin Thompson. 3: EM based on an algorithm by Esa Mäntysaari. 4: AI, but with step halving if an update goes outside the parameter space. Dense computation 31: AI, but combining AI and EM if an update goes outside the parameter space. 32: EM based on an algorithm by Robin Thompson. 33: EM based on an algorithm by Esa Mäntysaari. 34: AI, but with step halving if an update goes outside the parameter space. For task = 2 (RJMCL) method must be 0. Additional parameters for RJMC must be specified in the section $RJMC. For task = 11 (BLUP) method can be: Sparse computation 1: Jacobi Conjugate Gradient (JCG). 2: Jacobi Semi-Iteration (JSI). 3: Successive Overrelaxation (SOR). 4: Symmetric SOR Conjugate Gradient (SSORCG). 5: Symmetric SOR Semi-Iteration (SSORSI). 6: Reduced System Conjugate Gradient (RSCG). 7: Reduced System Semi-Iteration (RSSI). 8: FSPAK – Prediction error calculated one by one. 9: FSPAK – Prediction error calculated from a sparse inverse of the MME speed optimized. 10: FSPAK – Prediction error calculated from a sparse. inverse of the MME memory optimized. 21: PARDISO – Parallel solver on multi CPU and/or multi Core computers. Prediction error is not computed. Dense computation 30: MME build as dens matrix and solve using LAPACK subroutines. This require that MME have full rank. For task = 12 method can be: 2: Preconditioned Conjugate Gradient. scaling: = 0: No scaling of data prior to computation. = 1: Data are scaled by the specified residual variance ($PRIOR) before computations. This can help convergency in multitrait models, where the traits are on different scale. Estimated parameters and effects are scaled back to the original units. test_prt = 0: Standard. Yield minimum amount of output 1: Standard output plus lists of all class levels and with number of observations 2: As 1 plus additional test output. WARNING: this option may generate large volumes of output. .. _data: $DATA ----- .. code-block:: Data Description of data file. $DATA FMT (#int,#real,miss) fn [fn2] where: FMT = ASCII or BINARY #int = no. of integer variables #real = no. of real variables miss = reals below this value are regarded as missing fn = name of the data files. Starting with "/" => full path and name Otherwise relative to current directory fn2 = if specified, integer part is in fn, and real part is in fn2 Format of data file(s) see: section DATA FILE(S). .. _variable: $VARIABLE --------- .. code-block:: Variable Specifies names for the variables in the data set. The names can be up to 8 character long. If not specified variables are named I1-I#int and R1-R#real. $VARIABLE Followed by lines with names for all integer and real input variables in the data set. Variable names can be specified as individual names or as a indexed group of variable names using the following syntax: SNP[1:45000] This will create 45000 variable names: SNP1, SNP2, ..., SNP45000 .. _model: $MODEL ------ .. code-block:: Model Specifies the model or a file containing the model specifications. $MODEL [fn] where fn = name of file containing the model description. Starting with "/" => full path and name otherwise relative to current directory. Otherwise model directives are read from lines following the $MODEL keyword. Format of model directives see: section MODEL DIRECTIVES. .. _glmm: $GLMM ----- .. code-block:: Glmm Specifies that a trait is modeled by a Generalized Linear Mixed Model. $GLMM trait VARF=vf LINK=lf [OFFSET=ri] [CF=x] where trait = trait number (sequence) as specified in the $MODEL section vf = the variance function lf = The link function Optional an offset and a correction factor can be specified. ri = real input number for the offset variable x = real constant to add to data in order to avoid singularities in the initial iteration (default value = 0.5) Implemented variance functions: NORMAL, POISSON, BINOMIAL, GAMMA and INVGAUSSIAN Implemented link functions: IDENTITY, LOG, EXPONENTIAL, RECIPROCAL, LOGIT, PROBIT, COMPLOG and LOGLOG .. _glmm_pred: $GLMM_PRED ---------- .. code-block:: Glmm_pred Specifies iteration parameters for prediction in models with trait(s) modeled via GLMM (works only for DMU4). In order to improve convergency, GLMM_PRED starte with a GLM run on the fixed part of the model. When this has converged, GLMM iteration on the full model is started. $GLMM_PRED Round_Fixed CD_Fixed Round_Full CD_Full Where Round_Fixed = Max. no. of GLMM iterations on the fixed part of the model (integer > 1) CD_Fixed = Convergence criteria for the fixed part of the model (real < 1.0) Round_Full = Max. Nr. of GLMM iterations on the full model CD_Full = Convergence criteria for the full model .. _reduce: $REDUCE ------- .. code-block:: Reduce Used to merge random factors across trait, e.g. a random factor could be defined to have same effect on several traits. $REDUCE Followed by a line per trait. Each line contain as many numbers as there are random factors in the model (Co-variance matrices except the residual). For each random factor, the position in the (co)variance matrix must be specified, "0" indicate that this random factor is not in the model for the trait. Example: 4 trait and 3 random factors 1 1 1 1 2 1 1 3 2 1 4 2 The first random factor is specified to have the same effect on each trait. The second random factor is specified to have different effect on each trait. The third random factor is specified to have the same effect on trait 1 and 2, and a different on trait 3 and 4. .. _var_str: $VAR_STR -------- .. code-block:: Var_str Specify (co)variance structure for random factors. $VAR_STR r_factor type where : r_factor = structure number, used to associate (co)variance. structure to random effects in the model section. type = PED, DOM, COR, GREL, PGMIX, PGMIX_S, PGMIX_R, ABS_QTL or GROUP. Options for type = PED method = method for forming A-1 (1, 2, 3, or 6). [ RAM ] = if specified, a Reduced Animal Model relationship is used for sampling genetic dispersion parameters This works only for RJMC (task = 2). FMT = ASCII or BINARY fn = name of the pedigree file. Starting with "/" => full path and name otherwise relative to current directory. If method = 6, the PHG’s can be treated as fixed or random. In the latter case, the method (6) must be followed by the word “RANDOM” and a real number. The (co)-variance matrix multiplied by this number is added to the diagonal element/block for PHG’s. Options for type = DOM ass_rf = Random effect with the corresponding the corresponding pedigree structure. FMT = ASCII or BINARY. fn = name of the file with elements of the inverse dominance matrix. Starting with "/" => full path and name otherwise relative to current directory. Options for type = COR FMT = ASCII or BINARY. fn = name of the file with elements of the inverse co-variance matrixi. Starting with "/" => full path and name otherwise relative to current directory. Options for type = GREL FMT = ASCII or BINARY fn = name of the file with elements of the inverse co-variance matrix. Starting with "/" => full path and name otherwise relative to current directory. This option is for situations where the correlation structure is dens as in the case of genomic relationship It utilize dense matrix operation and can use parallel computation and is therefore much faster the COR option. Options for type = PGMIX method = method for forming A-1 (1, 2, 3, 4 or 6) FMT = ASCII or BINARY fn = name of the pedigree file. Starting with "/" => full path and name otherwise relative to current directory. typed = name of the file with ID’s of typed animals. Starting with "/" => full path and name otherwise relative to current directory. G-mat = name of the file with elements of the genomic relationship matrix. Starting with "/" => full path and name otherwise relative to current directory. w.w = optional weight to put on additive relationship matrix when forming the combined relationship matrix (see Christensen and Lund, 2010). G-ADJUST = Adjust elements in the genomic relationship so Average of diagonal elements and average of off-diagonal elements equal the same average in the additive relationship for the typed animals. Options for type = PGMIX_S Same format as PGMIX, but saves the part added to the inverse additive relationship matrix to construct the H-1 matrix. This is for facilitating the use of the same H-1 for several analysis. Options for type = PGMIX_R Same format as PGMIX, but reuse the saves part to added to the inverse additive relationship matrix to construct the H-1 matrix. Options for type = ABS_QTL ass_rf = Random effect with the same structure as the QTL effect (This will typically be the random effect with a pedigree structure). FMT = ASCII or BINARY. fn = name of the data files. Starting with "/" => full path and name otherwise relative to current directory. Options for type = GROUP This is for handling data with stratified heterogeneous residual (co)variance. Int. no. = Integer input no. for the variable relating observations to heterogeneous residual variance strata (group) Format of variance structure file see: VARIANCE STRUCTURE .. _var_rest: $VAR_REST --------- .. code-block:: Var_rest Specification of restrictions on (co)-variance matrices. $VAR_REST type options. where type = type of restriction. Type of restriction: VAR = Variance components are kept as specified as prior. COV = Co-variance components are kept as specified as prior. COR = Correlation is kept as specified as prior. V_RATIO = Variance ratios are kept as specified by the priors. To keep all variances in a co-variance matrix at the values specified in the $PRIOR section of the driver file, include the following line in the driver file: $VAR_REST VAR rf_no ALL Where rf_no = random factor number = co-variance matrix number. ALL = all variances should be kept at the specified value. If only some of the variances should be kept at the specified value: $VAR_REST VAR rf_no E r_no(1) r_no(2) ... r_no(n) where rf_no = random factor number = co-variance matrix number. E = only some of the variances that should be kept at the specified value. r_no(x) = row/column for the variance the keep constant ( 1 <= x <= the dimension of the co-variance matrix). To keep all co-variances in a co-variance matrix at the specified value by the following line: $VAR_REST COV rf_no ALL If only some co-variance components should be kept at the specified value specification of both row and column are needed, so use: $VAR_REST COV rf_no E r_no(1),c_no(1) r_no(2),c_no(2) ... r_no(n),c_no(n) Correlations can also be kept at the values specified by the priors. It is specified in the same way as keeping co-variance component constant, except that the key word “COV” is replaced by “COR”. $VAR_REST COR rf_no ALL $VAR_REST COR rf_no E r_no(1),c_no(1) r_no(2),c_no(2) ... r_no(n),c_no(n) Variance ratios can also be kept at the values specified by the priors. This requires specifications of which variance to restrict and the function of variances that should be kept constant (=the value specified by the $PRIOR section). This only works for functions of (co)-variance matrices of equal dimensions. If all variance ratios are to be kept constant specify: $VAR_REST V_RATIO rest_rf_no n_num rfn_no(1) .. rfn_no(n_num) n_den rfd_no(1) ..rfd_no(n_den) ALL where rest_rf_no = the number for the random factor ((co)variance matrix to restrict) n_num = number of co-variance matrices in the numerator rfn_no(i) = (co)-variance matrix i in the numerator (i=1,… , n_num) n_den = number of co-variance matrices in the denominator rfd_no(j) = (co)-variance matrix j in the denominator (j=1,.. , n_den) If only variance ratios for some of the elements are to be kept constant, “ALL” should be replaced by “E”, and the row/column number for the element(s) to impose restrictions on should be specified: $VAR_REST V_RATIO rest_rf_no n_num rfn_no(1) .. rfn_no(n_num) n_den rfd_no(1) .. rfd_no(n_den) E r_no(1) r_no(2)..r_no(n) .. _mixture: $MIXTURE -------- .. code-block:: Mixture Can only be used with the gibbs sampler (rjmc) module. It specifies that at least one trait is modelled as a mixture of two distributions $MIXTURE int. no. where : int_ no. = integer input no. for the variable contaning the initial guess for which distribution (1 or 2) the observation belongs to. This variable is updated in each round of the Gibbs sampler .. _prior: $PRIOR ------ .. code-block:: Prior Specifies priors / starting values / true values for (co)variance components or a file containing priors / starting values / true values. If not specified, an identity matrix is assumed for all (co)variance matrices for the model. For task = 11 and 12 all non-zero (co)-variance components must be specified. $PRIOR [fn] where fn = Starting with "/" => full path and name Otherwise relative to current directory If specified, priors are read from fn Otherwise priors are read from lines Following the $PRIOR keyword Format for priors see :VARIANCES AND COVARIANCES .. _precond: $PRECOND -------- .. code-block:: Precond Specifies the layout of the preconditioned matrix used by DMU5. The structure is defined for the following 3 parts of MME: 1) Fixed over all regressions 2) Fixed nested regressions 3) Fixed classification effects $PRECOND a b c Where: a, b and c describes the structure for each of the 3 groups. Legal combinations are: S S S -> All fixed effects across all traits as one block F F F -> Fixed overall regressions: Full block across traits Fixed nested regressions: Full block across traits Fixed class effects: Full block across traits F T T -> Fixed overall regressions: Full block across traits Fixed nested regressions: Trait block Fixed class effects: Trait block F D D -> Fixed overall regressions: Full block across traits Fixed nested regressions: Diagonal Fixed class effects: Diagonal T T T -> Fixed overall regressions: Trait block Fixed nested regressions: Trait block Fixed class effects: Trait block T D D -> Fixed overall regressions: Trait block Fixed nested regressions: Diagonal Fixed class effects: Diagonal D D D ->Fixed overall regressions: Diagonal Fixed nested regressions: Diagonal Fixed class effects: Diagonal .. _solution: $SOLUTION --------- .. code-block:: Solution Specify that the final solution vector is stored on disk and the fist max 250 solutions for each effect is printed to standard output. For the "free" version of DMU, only solutions for 1000 levels for random factors for which there are specified an variance structure are stored. In order to have the fulle solution vector, a license is needed .. _sol_cov: $SOL_COV -------- .. code-block:: Sol_cov Compute co-variances and correlations matrix between solutions to selected equations. The solutions, co-variances and correlations are stored in an ascii file with filename SOL_STD. The facility is only available in DMAUI and DMU4 when using direct solvers. The format is explained in the output to standard out. $SOL_PEV n_eq eq_1 eq_2 eq_3 …. Eq_neq Where n_eq is the number of selected equations and eq_1 eq_2 eg_3 … eq_n_eq are the equation number for which the (co-variances and correlations are required. .. _sol_pev: $SOL_PEV -------- .. code-block:: Sol_pev Compute and store the within random effect level solutions vector and the corresponding diagonal block of the inverse coefficient matrix (lower triangle format) in a file. The output is stored in an ascii file with filename SOL_PEV#, where # is the random effect number (rf_no). The format depends on the actual model and is explained in the output to standard out. The facility is only available in DMAUI and DMU4 when using direct solvers. For the "free" version of DMU, only solutions for 1000 levels for random factors for which there are specified an variance structure are stored. In order to have the fulle solution vector, a license is needed. $SOL_PEV r_factor where r_factor = structure number, used to associate (co)variance structure to random effects in the model section .. _residual: $RESIDUAL --------- .. code-block:: Residual Specifies that residuals should be computed and stored in a file. Computation of residuals is only implemented in DMUAI, DMU4 and DMU5 (Task 1, 11 and 12). $RESIDUALS FMT where: FMT = ASCII or BINARY. The content and format depends on the actual model and is explained in the output to standard out. .. _dmu4: $DMU4 ------ .. code-block:: Dmu4 Specifies optional input to DMU4 or a file containing the optional input. $DMU4 [fn] where fn = name of file contaning the optional input to DMU4. Starting with "/" => full path and name Otherwise relative to current directory. Otherwise input is read from lines following the $DMU4 keyword. Format of input see: OPTIONAL DMU4 INPUT .. _dmu5: $DMU5 ----- .. code-block:: Dmu5 Specifies optional input to DMU5 or a file containing the optional input. $DMU5 [fn] where fn = name of file contaning the optional input to DMU5. Starting with "/" => full path and name Otherwise relative to current directory. Otherwise input is read from lines following the $DMU5 keyword. Format of input see: OPTIONAL DMU5 INPUT .. _dmuai: $DMUAI ------ .. code-block:: Dmuai Specifies optional input to DMUAI or a file containing the optional input. $DMUAI [fn] where fn = name of file contaning the optional input to DMUAI. Starting with "/" => full path and name Otherwise relative to current directory. Otherwise input is read from lines following the $DMUAI keyword. Format of input see: OPTIONAL DMUAI INPUT. .. _rjmc: $RJMC ----- .. code-block:: Rjmc Specifies optional input to RJMC or a file containing the optional input. $RJMC [fn] where fn = name of file contaning the optional input to RJMC. Starting with "/" => full path and name Otherwise relative to current directory. Otherwise input is read from lines following the $RJMC keyword. Format of input see: OPTIONAL RJMC.