Skip to content

Commit 605ce13

Browse files
committed
docs: mention block
1 parent 723b1f8 commit 605ce13

File tree

1 file changed

+34
-11
lines changed

1 file changed

+34
-11
lines changed

guides/cheatsheets/axon_pytorch.cheatmd

Lines changed: 34 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ Applies a linear transformation to the incoming data: `y = xW^T + b`.
6060
#### Axon
6161
```elixir
6262
input = Axon.input("features")
63-
Axon.dense(input, out_features, activation: :relu, name: "my_dense_layer")
63+
dense_layer = Axon.dense(input, out_features, activation: :relu, name: "my_dense_layer")
6464
dense_layer = Axon.dense(input, 128)
6565

6666
#### PyTorch
@@ -77,10 +77,11 @@ Applies a 2D convolution over an input signal composed of several input planes.
7777
#### Axon
7878
```elixir
7979
# Example: 32 filters, 3x3 kernel, ReLU activation
80-
Axon.conv(32, kernel_size: 3, activation: :relu, padding: :same, name: "conv1")
80+
x = Axon.input("features")
81+
Axon.conv(x, 32, kernel_size: 3, activation: :relu, padding: :same, name: "conv1")
8182

8283
# Stride, padding, etc., are options:
83-
Axon.conv(64, kernel_size: {3, 3}, strides: 2, padding: :valid)
84+
Axon.conv(x, 64, kernel_size: {3, 3}, strides: 2, padding: :valid)
8485
```
8586

8687
*Note: Axon typically uses NHWC (Batch, Height, Width, Channels) format by default, common in TensorFlow/Keras.*
@@ -106,10 +107,10 @@ Applies 2D max pooling over an input signal.
106107
#### Axon
107108
```elixir
108109
# Example: 2x2 pool size, stride 2
109-
Axon.max_pool(kernel_size: 2, strides: 2, name: "pool1")
110+
Axon.max_pool(previous_layer, kernel_size: 2, strides: 2, name: "pool1")
110111

111112
# Padding can also be specified (default is :valid)
112-
Axon.max_pool(kernel_size: {3, 3}, strides: 1, padding: :same)
113+
Axon.max_pool(previous_layer, kernel_size: {3, 3}, strides: 1, padding: :same)
113114
```
114115

115116
*Note: Operates on NHWC format by default.*
@@ -145,7 +146,7 @@ Applies dropout during training (`mode: :train`). It's a no-op during inference
145146

146147
```elixir
147148
# Rate is the probability of an element being zeroed.
148-
Axon.dropout(rate: 0.5, name: "dropout1")
149+
Axon.dropout(previous_layer, rate: 0.5, name: "dropout1")
149150

150151
# Usage is implicit within the model's structure.
151152
# The mode (:train or :infer) is passed to the model execution function.
@@ -177,10 +178,10 @@ Normalizes the activations of the previous layer for each given example independ
177178
#### Axon
178179
```elixir
179180
# Typically applied to the feature dimension(s).
180-
Axon.layer_norm(name: "layernorm1")
181+
Axon.layer_norm(previous_layer, name: "layernorm1")
181182

182183
# Can specify the axis/axes for normalization (default is usually the last axis)
183-
# Axon.layer_norm(axis: -1, epsilon: 1.0e-5)
184+
# Axon.layer_norm(previous_layer, axis: -1, epsilon: 1.0e-5)
184185
```
185186

186187
#### PyTorch
@@ -216,10 +217,12 @@ model = Axon.input("input", shape: {nil, 784})
216217

217218
# Option 2: As separate layer
218219
model = Axon.input("input", shape: {nil, 10})
220+
|> Axon.dense(128)
219221
|> Axon.softmax()
220222

221223
# Common activation atoms: :relu, :softmax, :sigmoid, :tanh, :identity, etc.
222224
# Custom functions can also be used.
225+
# Axon.activation(layer, name) can also be used with the function name atoms.
223226
```
224227

225228
#### PyTorch
@@ -243,17 +246,37 @@ output = tanh(x)
243246
#### Axon
244247

245248
Axon allows for the definition of custom layers and models.
249+
`Axon.block/1` as shown below allows us to reuse the same parameters
250+
for an arbitrary Axon subgraph.
251+
252+
This means that the difference between the 2 examples below is that
253+
while the first has separate weights for the first and second dense layers,
254+
the second example uses the same weights for both.
246255

247256
```elixir
248257
# Example:
249-
defmodule MyCustomLayer do
250-
def call(x) do
258+
defmodule MyCustomLayers do
259+
def dense(x) do
251260
Axon.dense(x, 128, activation: :relu, name: "my_dense_layer")
252261
end
262+
263+
def block do
264+
Axon.block(&dense/1)
265+
end
253266
end
254267

255268
# Usage:
256-
model = Axon.input("input", shape: {nil, 784}) |> MyCustomLayer.call()
269+
input = Axon.input("input", shape: {nil, 784})
270+
model =
271+
input
272+
|> MyCustomLayers.dense()
273+
|> MyCustomLayers.dense()
274+
275+
dense_block = MyCustomLayers.block()
276+
model =
277+
input
278+
|> then(dense_block)
279+
|> then(dense_block)
257280
```
258281

259282
#### PyTorch

0 commit comments

Comments
 (0)