INFO:Output for tiny dataset:Epoch 0
INFO:Output for tiny dataset:	SGD with respect to sample 0
INFO:Output for tiny dataset:		Begin forward pass
INFO:Output for tiny dataset:			Value of a (before sigmoid):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset:			Value of z (after sigmoid):
[[1.  0.5 0.5 0.5 0.5]]
INFO:Output for tiny dataset:			Value of b (before softmax):
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]
INFO:Output for tiny dataset:			Value of y_hat (after softmax):
[[0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1]]
INFO:Output for tiny dataset:			Cross entropy: 2.3025850929940455
INFO:Output for tiny dataset:		Begin backward pass
INFO:Output for tiny dataset:			d(loss)/d(b):
[[ 0.1  0.1  0.1  0.1  0.1  0.1  0.1 -0.9  0.1  0.1]]
INFO:Output for tiny dataset:			d(loss)/d(beta):
[[ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]
 [-0.9  -0.45 -0.45 -0.45 -0.45]
 [ 0.1   0.05  0.05  0.05  0.05]
 [ 0.1   0.05  0.05  0.05  0.05]]
INFO:Output for tiny dataset:			d(loss)/d(z):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset:			d(loss)/d(a):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset:			d(loss)/d(alpha):
[[0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0.]]
INFO:Output for tiny dataset:		Update weights
INFO:Output for tiny dataset:			New alpha:
[[0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0.]]
INFO:Output for tiny dataset:			New beta:
[[-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [ 0.09   0.045  0.045  0.045  0.045]
 [-0.01  -0.005 -0.005 -0.005 -0.005]
 [-0.01  -0.005 -0.005 -0.005 -0.005]]
INFO:Output for tiny dataset:	SGD with respect to sample 1
INFO:Output for tiny dataset:		Begin forward pass
INFO:Output for tiny dataset:			Value of a (before sigmoid):
[[0. 0. 0. 0.]]
INFO:Output for tiny dataset:			Value of z (after sigmoid):
[[1.  0.5 0.5 0.5 0.5]]
INFO:Output for tiny dataset:			Value of b (before softmax):
[[-0.02 -0.02 -0.02 -0.02 -0.02 -0.02 -0.02  0.18 -0.02 -0.02]]
INFO:Output for tiny dataset:			Value of y_hat (after softmax):
[[0.09783393 0.09783393 0.09783393 0.09783393 0.09783393 0.09783393
  0.09783393 0.11949463 0.09783393 0.09783393]]
INFO:Output for tiny dataset:			Cross entropy: 2.324483831536847
INFO:Output for tiny dataset:		Begin backward pass
INFO:Output for tiny dataset:			d(loss)/d(b):
[[ 0.09783393  0.09783393  0.09783393  0.09783393  0.09783393  0.09783393
   0.09783393  0.11949463  0.09783393 -0.90216607]]
INFO:Output for tiny dataset:			d(loss)/d(beta):
[[ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [ 0.11949463  0.05974732  0.05974732  0.05974732  0.05974732]
 [ 0.09783393  0.04891696  0.04891696  0.04891696  0.04891696]
 [-0.90216607 -0.45108304 -0.45108304 -0.45108304 -0.45108304]]
INFO:Output for tiny dataset:			d(loss)/d(z):
[[0.00597473 0.00597473 0.00597473 0.00597473]]
INFO:Output for tiny dataset:			d(loss)/d(a):
[[0.00149368 0.00149368 0.00149368 0.00149368]]
INFO:Output for tiny dataset:			d(loss)/d(alpha):
[[0.00149368 0.         0.         0.         0.00149368 0.        ]
 [0.00149368 0.         0.         0.         0.00149368 0.        ]
 [0.00149368 0.         0.         0.         0.00149368 0.        ]
 [0.00149368 0.         0.         0.         0.00149368 0.        ]]
INFO:Output for tiny dataset:		Update weights
INFO:Output for tiny dataset:			New alpha:
[[-0.00014937  0.          0.          0.         -0.00014937  0.        ]
 [-0.00014937  0.          0.          0.         -0.00014937  0.        ]
 [-0.00014937  0.          0.          0.         -0.00014937  0.        ]
 [-0.00014937  0.          0.          0.         -0.00014937  0.        ]]
INFO:Output for tiny dataset:			New beta:
[[-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [ 0.07805054  0.03902527  0.03902527  0.03902527  0.03902527]
 [-0.01978339 -0.0098917  -0.0098917  -0.0098917  -0.0098917 ]
 [ 0.08021661  0.0401083   0.0401083   0.0401083   0.0401083 ]]
