---
title: "Multivariate Covariance Generalized Linear Models"
author: "Wagner Hugo Bonat"
date: "`r paste('mcglm', packageVersion('mcglm'), Sys.Date())`"
vignette: >
  %\VignetteIndexEntry{Multivariate Covariance Generalized Linear Models}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
##----------------------------------------------------------------------

library(knitr)

opts_chunk$set(
    dev.args=list(family="Palatino"))

options(width=68)

##----------------------------------------------------------------------

library(latticeExtra)
rm(list=ls())

## Color palette.
mycol <- c("#E41A1C", "#377EB8", "#4DAF4A", "#984EA3", "#FF7F00",
           "#FFFF33")
dput(mycol)

## Trellis graphical style.
ps <- list(
    box.rectangle=list(col=1, fill=c("gray70")),
    box.umbrella=list(col=1, lty=1),
    dot.symbol=list(col=1, pch=19),
    dot.line=list(col="gray50", lty=3),
    plot.symbol=list(col=1, cex=0.8),
    plot.line=list(col=1),
    plot.polygon=list(col="gray95"),
    superpose.line=list(col=mycol, lty=1),
    superpose.symbol=list(col=mycol, pch=1),
    superpose.polygon=list(col=mycol),
    strip.background=list(col=c("gray80","gray50"))
    )
trellis.par.set(ps)
## show.settings()

```

****

To install the stable version of [`mcglm`][], use
`devtools::install_git()`. For more information, visit [mcglm/README].

```{r, eval=FALSE}
library(devtools)
install_git("http://git.leg.ufpr.br/wbonat/mcglm.git")
```

```{r, eval=FALSE, error=FALSE, message=FALSE, warning=FALSE}
library(mcglm)
packageVersion("mcglm")
```

```{r, echo=FALSE, error=FALSE, message=FALSE, warning=FALSE}
library(mcglm)
packageVersion("mcglm")
```

##### Abstract
The `mcglm` package implements the multivariate covariance generalized
linear models (McGLMs) proposed by Bonat and J$\o$rgensen (2016).
The core fit function `mcglm` is employed for fitting a set of models. 
In this introductory vignette we restrict ourselves to model 
independent data, although a simple model for longitudinal data analysis
in the Gaussian case is also presented. We present models to deal with
continuous, binomial/bounded and count univariate response variables.
We explore the specification of different link, variance and covariance
functions.

****
## Regression models for continuous data

Consider a simple regression model, for univariate and independent 
Gaussian data:
$$Y \sim N(X \beta, \tau_0 Z_0).$$

```{r, warning = FALSE, message = FALSE}
# Loading extra packages
require(mcglm)
require(Matrix)
require(mvtnorm)
require(tweedie)

# Setting the seed
set.seed(2503)

# Fixed component
covariate <- seq(-1,1, l = 100)
X <- model.matrix(~ covariate)
mu1 <- mcglm::mc_link_function(beta = c(1,0.8), X = X, offset = NULL, 
                        link = "identity")
# Random component
y1 <- rnorm(100, mu1$mu, sd = 0.5)

# Matrix linear predictor
Z0 <- Diagonal(100, 1)

# Data set
data <- data.frame("y1" = y1, "covariate" = covariate)

# Fit
fit1.id <- mcglm(linear_pred = c(y1 ~ covariate), 
                 matrix_pred = list(list(Z0)),
                 data = data)

```

The `mcglm` package offers the following set of `S3-methods` for
model summarize.

```{r}
print(methods(class = "mcglm"))
```

The traditional `summary` for a fitted model can be obtained by

```{r}
summary(fit1.id)
```

The function `summary.mcglm` was designed to be similar the native
`summary` functions for the classes `lm` and `glm`. Extra features were
included to describe the dispersion structure. The mean formula call,
along with the the `link`, `variance` and `covariance` functions are
presented. The parameter estimates are presented in two blocks,
the first presents the regression estimates while the second presents
the dispersion estimates. In both cases the parameter estimates are
summarize by point estimates, standard errors and Z-values associated
with the Wald test whose null hypothesis is defined as $\beta = 0$ and
$\tau = 0$ for the mean and dispersion structures, respectively.
Finally, the selected algorithm, if the correction term is employed or 
not and the number of iterations is printed.

The same linear regresion model can be fitted by using a 
different covariance link function, for example the inverse covariance 
link function.

```{r}
# Fit using inverse covariance link function 
fit1.inv <- mcglm(linear_pred = c(y1 ~ covariate), 
                  matrix_pred = list(list(Z0)),
                  covariance = "inverse", data = data)

```
Furthermore, we can use a less conventional covariance link function,
as the exponential-matrix. 

```{r, message=FALSE, warning=FALSE}
# Fit using expm covariance link function
fit1.expm <- mcglm(linear_pred = c(y1 ~ covariate), 
                   matrix_pred = list(list(Z0)),
                   covariance = "expm", data = data)
```

The function `mcglm` returns an object of mcglm class, for which we can
use the method `coef` to extract the parameters estimates.

```{r}
# Comparing estimates using different covariance link functions
cbind(coef(fit1.id)$Estimates,
      coef(fit1.inv)$Estimates,
      coef(fit1.expm)$Estimates)

# Applying the inverse transformation
c(coef(fit1.id)$Estimates[3],
  1/coef(fit1.inv)$Estimates[3],
  exp(coef(fit1.expm)$Estimates[3]))
```

Consider an extension of the linear regression models to deal with 
heteroscedasticity:

$$ Y \sim N(X \beta, \tau_0 Z_0 + \tau_1 Z_1),$$
where $Z_0$ is a identity matrix and $Z_1$ is a diagonal matrix whose
elements are given by the values of a known covariate. Such a model,
can be fitted easily using the `mcglm` package.

```{r}
# Mean model
covariate <- seq(-1,1, l = 100)
X <- model.matrix(~ covariate)
mu1 <- mcglm::mc_link_function(beta = c(1,0.8), X = X, offset = NULL, 
                        link = "identity")

# Matrix linear predictor
Z0 <- Diagonal(100, 1)
Z1 <- Diagonal(100, c(rep(0,50),rep(1,50)))

# Covariance model
Sigma <- mcglm::mc_matrix_linear_predictor(tau = c(0.2, 0.15), 
                                           Z = list(Z0,Z1))
y1 <- rnorm(100, mu1$mu, sd = sqrt(diag(Sigma)))
data <- data.frame("y1" = y1, "covariate" = covariate)

fit2.id <- mcglm(linear_pred = c(y1 ~ covariate), 
                 matrix_pred = list(list(Z0,Z1)), 
                 data = data)
```

We can also extend the linear regression model to deal with longitudinal
data analysis. The code below presents an example of such a model.

```{r}
# Mean model
covariate <- seq(-1,1, l = 200)
X <- model.matrix(~ covariate)
mu1 <- mcglm::mc_link_function(beta = c(1,0.8), X = X, offset = NULL, 
                        link = "identity")
# Covariance model
Z0 <- Diagonal(200, 1)
Z1.temp <- Matrix(rep(1,10)%*%t(rep(1,10)))
Z1.list <- list()
for(i in 1:20){Z1.list[[i]] <- Z1.temp}
Z1 <- bdiag(Z1.list)
Sigma <- mcglm::mc_matrix_linear_predictor(tau = c(0.2, 0.15), Z = list(Z0,Z1))

# Response variable
y1 <- as.numeric(rmvnorm(1, mean = mu1$mu, sigma = as.matrix(Sigma)))
data <- data.frame("y1" = y1, "covariate" = covariate)

# Fit
fit3.id <- mcglm(linear_pred = c(y1 ~ covariate), 
                 matrix_pred = list("resp1" = list(Z0,Z1)), data = data)
```

The model summary

```{r}
summary(fit3.id)
```

Note that, the dispersion structure now has two parameters. 
In that case, the parameter $\tau_1$ represents the longitudinal 
structure for which we are assuming a compound symmetry model. This
model is an equivalent to a random intercept model in the context of
Linear Mixed Models (LMMs).

## Regression models for binomial and bounded data
The `mcglm` package offers a rich set of models to deal with binomial
and bounded response variables. The `logit`, `probit`, `cauchit`,
`cloglog`, and `loglog` link functions along with the extended binomial
variance function combined with the linear covariance structure, 
provide a flexible class of models for handling binomial and 
bounded response variables. The extended binomial variance function is
given by $\mu^p (1- \mu)^q$ where the two extra power parameters 
offer more flexibility to model the relationship between mean and 
variance. Consider the following simulated dataset.

```{r}
# Mean model
covariate <- seq(-1,1, l = 250)
X <- model.matrix(~ covariate)
mu1 <- mcglm::mc_link_function(beta = c(1,0.8), X = X, offset = NULL, 
                        link = "logit")
# Covariance model
Z0 <- Diagonal(500, 1)

# Response variable
set.seed(123)
y1 <- rbinom(500, prob = mu1$mu, size = 10)/10

# Data set
data <- data.frame("y1" = y1, "covariate" = covariate)
```

The most traditional regression model to deal with binomial data is the
logistic regression model that can be fitted using the `mcglm` package
using the following code:

```{r}
# Fit
fit4.logit <- mcglm(linear_pred = c(y1 ~ covariate), 
                    matrix_pred = list(list(Z0)),
                    link = "logit", variance = "binomialP",
                    power_fixed = TRUE,
                    Ntrial = list(rep(10,250)), data = data)
```

It is important to highlight that the response variable collumns should
be between $0$ and $1$ and in the case of more than one trial the 
argument `Ntrial` should be used for fitting the model. 
The argument `link` specifies the link function whereas the argument 
`variance` specifies the variance function in that case `binomialP`. 
The variance function `binomialP` represents a simplification of the 
extended binomial variance function given by $\mu^p (1- \mu)^p$. 
Note that, in this example the argument 'power_fixed = TRUE' specifies
that the power parameter $p$ will not be estimated, but fixed at the 
initial value $p = 1$ corresponding to the orthodox binomial variance
function. 
We can easily fit the model using a different link function, for example
the `cauchit`.

```{r}
fit4.cauchit <- mcglm(linear_pred = c(y1 ~ covariate), 
                      matrix_pred = list(list(Z0)),
                      link = "cauchit", variance = "binomialP", 
                      Ntrial = list(rep(10,250)), data = data)
```

We can also estimate the extra power parameter $p$.

```{r}
fit4.logitP <- mcglm(linear_pred = c(y1 ~ covariate), 
                      matrix_pred = list(list(Z0)),
                      link = "logit", variance = "binomialP",
                      power_fixed = FALSE,
                      Ntrial = list(rep(10,250)), data = data)
```

Furthermore, we can estimate the two extra power parameters involved in
the extended binomial variance function.

```{r}
fit4.logitPQ <- mcglm(linear_pred = c(y1 ~ covariate), 
                      matrix_pred = list(list(Z0)),
                      link = "logit", variance = "binomialPQ",
                      power_fixed = FALSE,
                      Ntrial = list(rep(10,250)), 
                      control_algorithm = list(tunning = 0.5),
                      data = data)
```

The estimation of the extra power parameters involved in the
extended binomial variance function is challenge mainly for small data 
sets. Note that, in this simulated example, we have to control the 
step-length of the `chaser` algorithm to avoid unrealistic values for
the parameters involved in the covariance structure. To do that, we used
the extra argument `control_algorithm` that should be a named list. 
For a detailed description of the arguments that can be passed to the
`control_algorithm` function see `?fit_mcglm`. 

## Regression models for count data

The analysis of count data in the `mcglm` package relies on the 
structure of the Poisson-Tweedie distribution. Such a distribution is
characterized by the following dispersion functions:

$$ \nu(\mu, p) = \mu + \tau_0 \mu^p. $$
The power parameter is an index that identify different distributions,
examples include the Hermite ($p = 0$), Neyman-Type A ($p = 1$) and the
negative binomial ($p = 2$). 

The orthodox Poisson model can be fitted using the `mcglm` package using
the `Tweedie` variance function $\nu(\mu, p ) = \mu^p$ where the power
parameter $p$ is fixed at $1$. For example,

```{r}
# Mean model
covariate <- seq(-2,2, l = 200)
X <- model.matrix(~ covariate)
mu <- mcglm::mc_link_function(beta = c(1,0.8), X = X, offset = NULL, 
                        link = "log")

# Covariance model
Z0 <- Diagonal(200, 1)

# Data
y1 <- rpois(200, lambda = mu$mu)
data <- data.frame("y1" = y1, "covariate" = covariate)

# Fit
fit.poisson <- mcglm(linear_pred = c(y1 ~ covariate), 
                    matrix_pred = list(list(Z0)),
                    link = "log", variance = "tweedie",
                    power_fixed = TRUE, data = data)
```

Another very useful model for count data is the negative binomial. 

```{r}
# Simulating negative binomial models
require(tweedie)
set.seed(1811)
y1 <- rtweedie(200, mu = mu$mu, power = 2, phi = 0.5)
y <- rpois(200, lambda = y1)
data <- data.frame("y1" = y, "covariate" = covariate)

fit.pt <- mcglm(linear_pred = c(y ~ covariate), 
               matrix_pred = list(list(Z0)),
               link = "log", variance = "poisson_tweedie", 
               power_fixed = FALSE, data = data)
summary(fit.pt)
```

Note that, we estimate the power parameter rather than fix it at $p = 2$.




<!---------------------------------------------------------------------- -->

[`mcglm`]: http://git.leg.ufpr.br/wbonat/mcglm
[mcglm/README]: http://git.leg.ufpr.br/wbonat/mcglm/blob/master/README.md
