PyTorch：nn

一个三阶多项式，训练用于根据 \(y=\sin(x)\) 从 \(-\pi\) 到 \(pi\) 预测 \(y=\sin(x)\)，方法是最小化欧几里得距离的平方。

此实现使用 PyTorch 中的 nn 包来构建网络。PyTorch autograd 使定义计算图和获取梯度变得容易，但是原始 autograd 对于定义复杂的神经网络来说可能有点太底层了；这就是 nn 包可以提供帮助的地方。nn 包定义了一组模块，您可以将它们视为产生输出的神经网络层，并且可能有一些可训练的权重。

99 240.86277770996094
199 167.79660034179688
299 117.86258697509766
399 83.69758605957031
499 60.295040130615234
599 44.246192932128906
699 33.227760314941406
799 25.654376983642578
899 20.443077087402344
999 16.853191375732422
1099 14.37750244140625
1199 12.668405532836914
1299 11.487274169921875
1399 10.670136451721191
1499 10.104262351989746
1599 9.71200942993164
1699 9.439839363098145
1799 9.25082015991211
1899 9.119425773620605
1999 9.028006553649902
Result: y = 0.013691586442291737 + 0.8503276705741882 x + -0.002362025436013937 x^2 + -0.09241817146539688 x^3

import torch
import math


# Create Tensors to hold input and outputs.
x = torch.linspace(-math.pi, math.pi, 2000)
y = torch.sin(x)

# For this example, the output y is a linear function of (x, x^2, x^3), so
# we can consider it as a linear layer neural network. Let's prepare the
# tensor (x, x^2, x^3).
p = torch.tensor([1, 2, 3])
xx = x.unsqueeze(-1).pow(p)

# In the above code, x.unsqueeze(-1) has shape (2000, 1), and p has shape
# (3,), for this case, broadcasting semantics will apply to obtain a tensor
# of shape (2000, 3)

# Use the nn package to define our model as a sequence of layers. nn.Sequential
# is a Module which contains other Modules, and applies them in sequence to
# produce its output. The Linear Module computes output from input using a
# linear function, and holds internal Tensors for its weight and bias.
# The Flatten layer flatens the output of the linear layer to a 1D tensor,
# to match the shape of `y`.
model = torch.nn.Sequential(
    torch.nn.Linear(3, 1),
    torch.nn.Flatten(0, 1)
)

# The nn package also contains definitions of popular loss functions; in this
# case we will use Mean Squared Error (MSE) as our loss function.
loss_fn = torch.nn.MSELoss(reduction='sum')

learning_rate = 1e-6
for t in range(2000):

    # Forward pass: compute predicted y by passing x to the model. Module objects
    # override the __call__ operator so you can call them like functions. When
    # doing so you pass a Tensor of input data to the Module and it produces
    # a Tensor of output data.
    y_pred = model(xx)

    # Compute and print loss. We pass Tensors containing the predicted and true
    # values of y, and the loss function returns a Tensor containing the
    # loss.
    loss = loss_fn(y_pred, y)
    if t % 100 == 99:
        print(t, loss.item())

    # Zero the gradients before running the backward pass.
    model.zero_grad()

    # Backward pass: compute gradient of the loss with respect to all the learnable
    # parameters of the model. Internally, the parameters of each Module are stored
    # in Tensors with requires_grad=True, so this call will compute gradients for
    # all learnable parameters in the model.
    loss.backward()

    # Update the weights using gradient descent. Each parameter is a Tensor, so
    # we can access its gradients like we did before.
    with torch.no_grad():
        for param in model.parameters():
            param -= learning_rate * param.grad

# You can access the first layer of `model` like accessing the first item of a list
linear_layer = model[0]

# For linear layer, its parameters are stored as `weight` and `bias`.
print(f'Result: y = {linear_layer.bias.item()} + {linear_layer.weight[:, 0].item()} x + {linear_layer.weight[:, 1].item()} x^2 + {linear_layer.weight[:, 2].item()} x^3')