Posted on Categories:CS代写, Machine Learning, 数据分析信号处理和机器学习中的矩阵方法, 机器学习, 计算机代写

## avatest™帮您通过考试

avatest™的各个学科专家已帮了学生顺利通过达上千场考试。我们保证您快速准时完成各时长和类型的考试，包括in class、take home、online、proctor。写手整理各样的资源来或按照您学校的资料教您，创造模拟试题，提供所有的问题例子，以保证您在真实考试中取得的通过率是85%以上。如果您有即将到来的每周、季考、期中或期末考试，我们都能帮助您！

•最快12小时交付

•200+ 英语母语导师

•70分以下全额退款

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Smooth functions

We can obtain better results for more restrictive classes of functions. Recall that we can only set a small step because the gradient changes as we change the solution. A function is smooth if the change in the gradient is bounded by how far the solution changes. More precisely, a function is $\beta$-smooth if for any two points $x, y$, we have
$$|\nabla f(x)-\nabla f(y)| \leq \beta|x-y|$$
Note that this implies for any $x, y$
$$f(y) \leq f(x)+\langle\nabla f(x), y-x\rangle+\frac{\beta}{2}|y-x|^2$$
Thus, we can always bound the objective using a quadratic function. In order to find the next location for the solution, we minimize the quadratic approximation at the current point. The solution turns out to be $\eta_t=1 / \beta$. With this choice, we have
$$f\left(x^{(t)}\right) \leq f\left(x^{(t-1)}\right)-\frac{1}{2 \beta} | \nabla f\left(x^{(t-1)} |^2\right.$$
We analyze this algorithm using the potential $\Phi_t=t\left(f\left(x^{(t)}\right)-f\left(x^\right)\right)+\frac{\beta}{2}\left|x^{(t)}-x^\right|^2$. The change in the potential is
$$\Phi_t-\Phi_{t-1}=t\left(f\left(x^{(t)}\right)-f\left(x^{(t-1)}\right)\right)+\left(f\left(x^{(t-1)}\right)-f\left(x^\right)\right)+\frac{\beta}{2}\left(\left|x^{(t)}-x^\right|^2-\left|x^{(t-1)}-x^*\right|^2\right)$$

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Constrained optimization

We now extend our results to the constrained case $\min _{x \in S} f(x)$ for a convex set $S$. In the unconstrained case, we take the quadratic approximation of the function at the current solution and the next solution is the minimizer of the quadratic approximation. Notice that this step is still meaningful when we have constraints. Thus our algorithm is
$$x^{(t)} \leftarrow \underset{z \in S}{\operatorname{argmin}} f\left(x^{(t-1)}\right)+\left\langle\nabla f\left(x^{(t-1)}\right), z-x^{(t-1)}\right\rangle+\frac{\beta}{2}\left|z-x^{(t-1)}\right|^2$$
Another idea is to move in the direction of the gradient, which might take us out of the feasible region, and then project back to the feasible region. In other words, our algorithm is
\begin{aligned} &y^{(t)} \leftarrow x^{(t-1)}-\eta_t \nabla f\left(x^{(t-1)}\right) \ &x^{(t)} \leftarrow \underset{x \in S}{\operatorname{argmin}}\left|y^{(t)}-x\right| \end{aligned}
It turns out that for step size $\eta_t=1 / \beta$, these two algorithms are identical. In order to analyze this algorithm, we need a property of the projection operation.

Lemma 6.1. Given a convex set $S$, let $a \in S$ and $b^{\prime} \in \mathbb{R}^n$. Let $b=\operatorname{argmin}_{x \in S} \frac{1}{2}\left|x-b^{\prime}\right|^2$. Then $\left\langle a-b, b-b^{\prime}\right\rangle \geq 0$ and therefore, $|a-b|^2 \leq\left|a-b^{\prime}\right|^2$.

Proof. The lemma follows from the optimality of $b$. The gradient of $\frac{1}{2}\left|x-b^{\prime}\right|^2$ at $x=b$ is $b-b^{\prime}$. Because of the optimality of $b$, we have $\left\langle a-b, b-b^{\prime}\right\rangle \geq 0$.

Using this property, we obtain $\left|x^{(t)}-x^\right|^2 \leq\left|y^{(t)}-x^\right|^2$ and we can observe that the rest of the original proof goes through in the constrained setting.

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代 考|Smooth functions

$$|\nabla f(x)-\nabla f(y)| \leq \beta|x-y|$$

$$f(y) \leq f(x)+\langle\nabla f(x), y-x\rangle+\frac{\beta}{2}|y-x|^2$$

$$f\left(x^{(t)}\right) \leq f\left(x^{(t-1)}\right)-\frac{1}{2 \beta} \mid \nabla f\left(\left.x^{(t-1)}\right|^2\right.$$

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。

Posted on Categories:CS代写, Machine Learning, 数据分析信号处理和机器学习中的矩阵方法, 机器学习, 计算机代写

## avatest™帮您通过考试

avatest™的各个学科专家已帮了学生顺利通过达上千场考试。我们保证您快速准时完成各时长和类型的考试，包括in class、take home、online、proctor。写手整理各样的资源来或按照您学校的资料教您，创造模拟试题，提供所有的问题例子，以保证您在真实考试中取得的通过率是85%以上。如果您有即将到来的每周、季考、期中或期末考试，我们都能帮助您！

•最快12小时交付

•200+ 英语母语导师

•70分以下全额退款

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Exact line search

We could also choose step to do the best we can along direction of negative gradient, called exact line search:
$$t=\underset{s \geq 0}{\operatorname{argmin}} f(x-s \nabla f(x))$$
Usually not possible to do this minimization exactly
Approximations to exact line search are typically not as efficient as backtracking, and it’s typically not worth it

Convergence analysis
Assume that $f$ convex and differentiable, with $\operatorname{dom}(f)=\mathbb{R}^n$, and additionally that $\nabla f$ is Lipschitz continuous with constant $L>0$,
$$|\nabla f(x)-\nabla f(y)|_2 \leq L|x-y|_2 \text { for any } x, y$$
(Or when twice differentiable: $\nabla^2 f(x) \preceq L I$ )
Theorem: Gradient descent with fixed step size $t \leq 1 / L$ satisfies
$$f\left(x^{(k)}\right)-f^{\star} \leq \frac{\left|x^{(0)}-x^{\star}\right|_2^2}{2 t k}$$
and same result holds for backtracking, with $t$ replaced by $\beta / L$
We say gradient descent has convergence rate $O(1 / k)$. That is, it finds $\epsilon$-suboptimal point in $O(1 / \epsilon)$ iterations

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Analysis for strong convexity

Reminder: strong convexity of $f$ means $f(x)-\frac{m}{2}|x|_2^2$ is convex for some $m>0$ (when twice differentiable: $\nabla^2 f(x) \succeq m I$ )

Assuming Lipschitz gradient as before, and also strong convexity:
Theorem: Gradient descent with fixed step size $t \leq 2 /(m+L)$ or with backtracking line search search satisfies
$$f\left(x^{(k)}\right)-f^{\star} \leq \gamma^k \frac{L}{2}\left|x^{(0)}-x^{\star}\right|_2^2$$
where $0<\gamma<1$
Rate under strong convexity is $O\left(\gamma^k\right)$, exponentially fast! That is, it finds $\epsilon$-suboptimal point in $O(\log (1 / \epsilon))$ iterations

Called linear convergence
Objective versus iteration curve looks linear on semilog plot

Important note: $\gamma=O(1-m / L)$. Thus we can write convergence rate as
$$O\left(\frac{L}{m} \log (1 / \epsilon)\right)$$
Higher condition number $L / m \Rightarrow$ slower rate. This is not only true of in theory … very apparent in practice too

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Exact line search

$$t=\underset{s \geq 0}{\operatorname{argmin}} f(x-s \nabla f(x))$$

$$|\nabla f(x)-\nabla f(y)|_2 \leq L|x-y|_2 \text { for any } x, y$$
(或者当两次可微时: $\nabla^2 f(x) \preceq L I$ )

$$f\left(x^{(k)}\right)-f^{\star} \leq \frac{\left|x^{(0)}-x^{\star}\right|_2^2}{2 t k}$$

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代 考|Analysis for strong convexity

$$f\left(x^{(k)}\right)-f^{\star} \leq \gamma^k \frac{L}{2}\left|x^{(0)}-x^{\star}\right|_2^2$$

$$O\left(\frac{L}{m} \log (1 / \epsilon)\right)$$

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。

Posted on Categories:CS代写, Machine Learning, 数据分析信号处理和机器学习中的矩阵方法, 机器学习, 计算机代写

## avatest™帮您通过考试

avatest™的各个学科专家已帮了学生顺利通过达上千场考试。我们保证您快速准时完成各时长和类型的考试，包括in class、take home、online、proctor。写手整理各样的资源来或按照您学校的资料教您，创造模拟试题，提供所有的问题例子，以保证您在真实考试中取得的通过率是85%以上。如果您有即将到来的每周、季考、期中或期末考试，我们都能帮助您！

•最快12小时交付

•200+ 英语母语导师

•70分以下全额退款

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Directional Derivative

Directional Derivative

Directional derivative in direction $u$ (a unit vector) is the slope of function $f$ in direction $u$

This evaluates to
$$u^T \nabla_x f(x)$$

Example: let $u^T=\left(u_x, u_y, u_z\right)$ be a unit vector in Cartesian coordinates, so then
$$|u|_2=\sqrt{u_x^2+u_y^2+u_z^2}=1$$
$$u^T \nabla_x f(x)=\frac{\partial f}{\partial x} u_x+\frac{\partial f}{\partial y} u_y+\frac{\partial f}{\partial z} u_z$$

Directional Derivative

To minimize $f$ find direction in which $f$ decreases the fastest
$$\min {u, u^T u=1} u^T \nabla_x f(x)=\min {u, u^T u=1}|u|_2 \cdot\left|\nabla_x f(x)\right|_2 \cdot \cos \theta$$

where $\theta$ is angle between $u$ and the gradient

Substitute $|u|_2=1$ and ignore factors that not depend on $u$ this simplifies to
$$\min _u \cos \theta$$

This is minimized when $u$ points in direction opposite to gradient

In other words, the gradient points directly uphill, and the negative gradient points directly downhill

## 计算机代写|数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Method of Gradient Descent

The gradient points directly uphill, and the negative gradient points directly downhill

Thus we can decrease $f$ by moving in the direction of the negative gradient

This is known as the method of steepest descent or gradient descent

Steepest descent proposes a new point
$$x^{\prime}=x-\epsilon \nabla_x f(x)$$

where $\epsilon$ is the learning rate, a positive scalar. Set to a small constant.

Choosing $\epsilon$ : Line Search

We can choose $\epsilon$ in several different ways

Popular approach: set $\epsilon$ to a small constant

Another approach is called line search:

Evaluate
$$f\left(x-\epsilon \nabla_x f(x)\right)$$
for several values of $\epsilon$ and choose the one that results in smallest objective function value

## 计算机代写数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Directional Derivative

$$u^T \nabla_x f(x)$$

$$\begin{gathered} |u|_2=\sqrt{u_x^2+u_y^2+u_z^2}=1 \ u^T \nabla_x f(x)=\frac{\partial f}{\partial x} u_x+\frac{\partial f}{\partial y} u_y+\frac{\partial f}{\partial z} u_z \end{gathered}$$

$$\min u, u^T u=1 u^T \nabla_x f(x)=\min u, u^T u=1|u|_2 \cdot\left|\nabla_x f(x)\right|_2 \cdot \cos \theta$$

$$\min _u \cos \theta$$

## 计算机代写数据分析信号处理和机器学习中的矩阵方法代写Matrix Methods In Data Analysis, Signal Processing, And Machine Learning代考|Method of Gradient Descent

$$x^{\prime}=x-\epsilon \nabla_x f(x)$$

$$f\left(x-\epsilon \nabla_x f(x)\right)$$

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。