【Fundamentals】Precise Analysis of Multivariate Linear Regression and the Regress Function in MATLAB

# **1. Overview of Multiple Linear Regression**

Multiple linear regression is a statistical modeling technique used to predict the linear relationship between a continuous dependent variable (target variable) and multiple independent variables (predictor variables). Unlike simple linear regression, multiple linear regression allows the model to include several independent variables, thus providing a more comprehensive description of the dependent variable's variation.

The mathematical form of a multiple linear regression model is:

y = β0 + β1x1 + β2x2 + ... + βnxn + ε

Where:

* y is the dependent variable
* x1, x2, ..., xn are independent variables
* β0, β1, ..., βn are model parameters
* ε is the error term

# **2. Theory of Multiple Linear Regression**

Multiple linear regression is a statistical modeling technique used to predict the linear relationship between one or more independent variables (explanatory variables) and a dependent variable (response variable). It extends simple linear regression by allowing multiple independent variables to be considered simultaneously.

**2.1 Linear Regression Model**

**2.1.1 Model Establishment**

The mathematical form of a multiple linear regression model is as follows:

y = β0 + β1x1 + β2x2 + ... + βnxn + ε

Where:

* y is the dependent variable
* x1, x2, ..., xn are independent variables
* β0 is the intercept term
* β1, β2, ..., βn are the regression coefficients of the independent variables
* ε is the error term

**2.1.2 Parameter Estimation**

Regression coefficients β can be estimated using the least squares method, i.e., finding the coefficients that minimize the sum of squared errors (SSE). The SSE is defined as:

SSE = Σ(yi - ŷi)^2

Where:

* yi is the actual value of the dependent variable
* ŷi is the predicted value of the dependent variable

**2.2 Model Evaluation**

**2.2.1 Goodness of Fit**

***mon indicators include:

* Coefficient of determination (R^2): Represents the percentage of data variation explained by the model.
* Adjusted coefficient of determination (Adjusted R^2): Considers the effect of the number of independent variables on R^2.

**2.2.2 Predictive Ability**

***mon indicators include:

* Root Mean Square Error (RMSE): Represents the average difference between predicted values and actual values.
* Mean Absolute Error (MAE): Represents the average absolute difference between predicted values and actual values.

**2.3 Hypothesis Testing**

**2.3.1 Significance Testing of Parameters**

Significance testing of parameters is used to determine if independent variables significantly affect the dependent variable. T-tests and p-values are used to assess the significance of each regression coefficient.

**2.3.2 Significance Testing of the Model**

Model significance testing is used to determine if the entire model significantly affects the data. F-tests and p-values are used to assess the overall fit of the model.

# **3. Practice of Multiple Linear Regression**

### **3.1 Data Preparation**

#### **3.1.1 Data Collection**

The establishment of a multiple linear regression model requires relevant data collection. The sources of data collection can be internal data, external data, or a combination of both.

**Internal Data:** Data from internal databases, business systems, or other data sources within the enterprise. Examples include sales data, customer data, production data, etc.

**External Data:** Data from public datasets, market research, or other external sources. Examples include industry reports, demographic statistics, economic indicators, etc.

#### **3.1.2 Data Preprocessing**

Data collected usually requires preprocessing to ensure data quality and usability. The main steps of data preprocessing include:

***Data Cleaning:** Removing missing values, outliers, and erroneous data.
***Data Transformation:** Transforming data into a format suitable for model analysis, such as standardization or normalization.
***Feature Engineering:** Creating new features or transforming existing features to improve the model's predictive power.

### **3.2 Model Establishment**

#### **3.2.1 Use of the regress Function**

In MATLAB, the `regress` function can be used to establish a multiple linear regression model. The syntax of the `regress` function is as follows:

[b, bint, r, rint, stats] = regress(y, X)

Where:

* `y`: Dependent variable vector
* `X`: Independent variable matrix
* `b`: Regression coeffici