Posted on Categories:Computer Vision, 计算机代写, 计算机视觉

# 计算机代写|计算机视觉代写Computer Vision代考|CITS4402 Second-Order Optimization

avatest™

## avatest™帮您通过考试

avatest™的各个学科专家已帮了学生顺利通过达上千场考试。我们保证您快速准时完成各时长和类型的考试，包括in class、take home、online、proctor。写手整理各样的资源来或按照您学校的资料教您，创造模拟试题，提供所有的问题例子，以保证您在真实考试中取得的通过率是85%以上。如果您有即将到来的每周、季考、期中或期末考试，我们都能帮助您！

•最快12小时交付

•200+ 英语母语导师

•70分以下全额退款

## 计算机代写|计算机视觉代写Computer Vision代考|Newton’s Method

Newton’s method is a classical example of second-order continuous optimization (see, e.g., [2] for a more detailed description). Here, the function $f(\mathbf{x})$ is approximated by a second-order Taylor expansion $T$ at the current solution $\mathbf{x}^k$ :
$$f(\mathbf{x}) \cong T(\delta \mathbf{x})=f\left(\mathbf{x}^k\right)+\nabla f\left(\mathbf{x}^k\right) \cdot \delta \mathbf{x}+\frac{1}{2} \delta \mathbf{x}^T \cdot \mathbf{H}\left(\mathbf{x}^k\right) \cdot \delta \mathbf{x}$$
with $\delta \mathbf{x}$ being the difference $\mathbf{x}-\mathbf{x}^k$. As long as $\delta \mathbf{x}$ remains sufficiently small, we can be quite sure that the second-order Taylor expansion $T(\delta \mathbf{x})$ is a sufficiently good approximation of $f(\mathbf{x})$.

As $f(\mathbf{x})$ is approximated by a quadratic form, a candidate of its minimum can be found analytically in a single step by setting the derivative of the quadratic form to zero. This yields a linear system of equations which can be solved with standard techniques (see also Sect. 2.1). Because the Taylor expansion is just an approximation of $f(\mathbf{x})$, its minimization at a single position is usually not sufficient for finding the desired solution. Hence, finding a local minimum of $f(\mathbf{x})$ involves an iterative application of the following two steps:

1. Approximate $f(\mathbf{x})$ by a second-order Taylor expansion $T(\delta \mathbf{x})$ (see (2.16)).
2. Calculate the minimizing argument $\delta \mathbf{x}^*$ of this approximation $T(\delta \mathbf{x})$ by setting its first derivative to zero: $\nabla T(\delta \mathbf{x})=\mathbf{0}$

In order for $\delta \mathbf{x}^*$ to be a local minimum of $T(\delta \mathbf{x})$, the following two conditions must hold:

1. $\nabla T\left(\delta \mathbf{x}^*\right)=\mathbf{0}$.
2. $\mathbf{H}\left(\mathbf{x}^k\right)$ is positive, i.e., $\mathbf{d}^T \cdot \mathbf{H}\left(\mathbf{x}^k\right) \cdot \mathbf{d}>0$ for every vector $\mathbf{d}$. This is equivalent to the statement that all eigenvalues of $\mathbf{H}\left(\mathbf{x}^k\right)$ are positive real numbers. This condition corresponds to the fact that for one-dimensional functions, their second derivative has to be positive at a local minimum.

Now let’s see how the two steps of Newton’s method can be implemented in practice. First, a differentiation of $T(\delta \mathbf{x})$ with respect to $\delta \mathbf{x}$ yields:
$$\nabla f(\mathbf{x}) \cong \nabla T(\delta \mathbf{x})=\nabla f\left(\mathbf{x}^k\right)+\mathbf{H}\left(\mathbf{x}^k\right) \cdot \delta \mathbf{x}$$

## 计算机代写|计算机视觉代写Computer Vision代考|Gauss-Newton and Levenberg-Marquardt Algorithm

A special case occurs if the objective function $f(\mathbf{x})$ is composed of a sum of squared values:
$$f(\mathbf{x})=\sum_{i=1}^N r_i(\mathbf{x})^2$$

Such a specific structure of the objective can be encountered, e.g., in least squares problems, where the $r_i(\mathbf{x})$ are deviations from the values of a regression function to observed data values (so-called residuals). There are numerous vision applications where we want to calculate some coefficients $\mathbf{x}$ such that the regression function fits “best” to the sensed data in a least squares sense.

If the residuals are linear in $\mathbf{x}$, we can apply the linear regression method already presented in Sect. 2.1. Nonlinear $r_i(\mathbf{x})$, however, are a generalization of this regression problem and need a different proceeding to be solved.

Please bear in mind that in order to obtain a powerful method, we always should utilize knowledge about the specialties of the problem at hand if existent. The Gauss-Newton algorithm (see, e.g., [1]) takes advantage of the special structure of $f(\mathbf{x})$ (i.e., $f(\mathbf{x})$ is composed of a sum of residuals) by approximating the secondorder derivative by first-order information.

To understand this, let’s examine how the derivatives used in Newton’s method can be written for squared residuals. Applying the chain rule, the elements of the gradient $\nabla f(\mathbf{x})$ can be written as
$$\nabla f_j(\mathbf{x})=2 \sum_{i=1}^N r_i(\mathbf{x}) \cdot J_{i j}(\mathbf{x}) \quad \text { with } \quad J_{i j}(\mathbf{x})=\frac{\partial r_i(\mathbf{x})}{\partial x_j}$$
where the $J_{i j}(\mathbf{x})$ are the elements of the so-called Jacobi matrix $\mathbf{J}{\mathbf{r}}(\mathbf{x})$, which pools first-order derivative information of the residuals. With the help of the product rule, the Hessian $\mathbf{H}$ can be derived from $\nabla f(\mathbf{x})$ as follows: \begin{aligned} H{j l}(\mathbf{x}) &=\frac{\nabla f_j(\mathbf{x})}{\partial x_l}=2 \sum_{i=1}^N\left(\frac{\partial r_i(\mathbf{x})}{\partial x_l} \cdot J_{i j}(\mathbf{x})+r_i(\mathbf{x}) \cdot \frac{\partial J_{i j}(\mathbf{x})}{\partial x_l}\right) \ &=2 \sum_{i=1}^N\left(J_{i l}(\mathbf{x}) \cdot J_{i j}(\mathbf{x})+r_i(\mathbf{x}) \cdot \frac{\partial^2 r_i(\mathbf{x})}{\partial x_j \cdot \partial x_l}\right) \end{aligned}

## 计算机代写计算机视觉代写Computer Vision代考|Newton’s Method

$$f(\mathbf{x}) \cong T(\delta \mathbf{x})=f\left(\mathbf{x}^k\right)+\nabla f\left(\mathbf{x}^k\right) \cdot \delta \mathbf{x}+\frac{1}{2} \delta \mathbf{x}^T \cdot \mathbf{H}\left(\mathbf{x}^k\right) \cdot \delta \mathbf{x}$$

$\nabla T\left(\delta \mathbf{x}^*\right)=\mathbf{0}$.

$\mathbf{H}\left(\mathbf{x}^k\right)$ 为正，即 $\mathbf{d}^T \cdot \mathbf{H}\left(\mathbf{x}^k\right) \cdot \mathbf{d}>0$ 对于每个向量 $\mathrm{d}$. 这等价于声明所有的特征值 $\mathbf{H}\left(\mathbf{x}^k\right)$ 是正实数。这个条件对应于这 样一个事实: 对于一维函数，它们的二阶导数必须在局部最小值处为正。

$$\nabla f(\mathbf{x}) \cong \nabla T(\delta \mathbf{x})=\nabla f\left(\mathbf{x}^k\right)+\mathbf{H}\left(\mathbf{x}^k\right) \cdot \delta \mathbf{x}$$

## 计算机代写|计算机视觉代写Computer Vision代考|Gauss-Newton and Levenberg-Marquardt Algorithm

$$f(\mathbf{x})=\sum_{i=1}^N r_i(\mathbf{x})^2$$

$$\nabla f_j(\mathbf{x})=2 \sum_{i=1}^N r_i(\mathbf{x}) \cdot J_{i j}(\mathbf{x}) \quad \text { with } \quad J_{i j}(\mathbf{x})=\frac{\partial r_i(\mathbf{x})}{\partial x_j}$$

$$H j l(\mathbf{x})=\frac{\nabla f_j(\mathbf{x})}{\partial x_l}=2 \sum_{i=1}^N\left(\frac{\partial r_i(\mathbf{x})}{\partial x_l} \cdot J_{i j}(\mathbf{x})+r_i(\mathbf{x}) \cdot \frac{\partial J_{i j}(\mathbf{x})}{\partial x_l}\right) \quad=2 \sum_{i=1}^N\left(J_{i l}(\mathbf{x}) \cdot J_{i j}(\mathbf{x})+r_i(\mathbf{x}) \cdot \frac{\partial^2 r_i(\mathbf{x})}{\partial x_j \cdot \partial x_l}\right)$$

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。