Skip to main content

Machine Learning

Search This Blog

Backpropagation Algorithm 2

Get link
Facebook
X
Pinterest
Email
Other Apps

Backpropagation Algorithm

\(g'(z^{(3)})\) = derivative of activation function g evaluate at \(z^{(3)}\)

Resources:
- https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions
- https://www.coursera.org/learn/machine-learning/supplement/pjdBA/backpropagation-algorithm
- https://www.coursera.org/learn/machine-learning/supplement/v5Bu8/backpropagation-intuition

Backpropagation Algorithm

Get link
Facebook
X
Pinterest
Email
Other Apps

Comments

Post a Comment

Popular Posts

Multiclass Classification: One-vs-all

Multiclass Classification: One-vs-all Classify more than just 0 and 1. Examples: Email foldering/tagging: Work (y=1), Friends (2), Family (3), Hobby (4) Medical diagrams: Not ill (1), Cold (2), Flu (3) Weather: Sunny (1), Cloudy (2), Rain (3), Snow (4) Summary by Filip: Logistic regression is a classification algorithm and not a regression algorithm, although "regression" is part of the name Hypothesis equals sigmoid/logistic function of theta_transpose * x The derivative of J(theta) is the same as for linear regression, i.e. 1/m * sum((h(x) - y)*x). However, the hypothesis is different. While the hypothesis in linear regression is h(x) = theta_transpose * X , the hypothesis in logistic regression is 1/(1+e^(theta_transpose * x)) Function z does not have to be linear; it can be everything, e.g. circle When dealing with multiclass logistic regression, if we follow the one vs. all approach, we can simplify the problem to simple binary classification problem....

Octave Cheatsheet

Octave Commands: Arithmetics: +, -, *, /, ^ Logical operations: ==, ~= (not equal), &&, ||, xor Change prompt: PS1('>> '); Semicolon: suppressing output A = [1 2; 3 4; 5 6] - a 3x2 matrix B = [1; 2; 3; 4; 5] - a 5x1 matrix v = 1:0.1:2 - row matrix from 1 to 2 with 0.1 increment ones(2, 3) - 2x3 matrix of all 1s zeros(1,3) - 1x3 matrix of all 0s w = rand(1,3) - 1x3 matrix of all random numbers hist(w) - histogram eye (4) - 4x4 identity matrix help eye - how to use 'eye' function size(A) = [2, 3] - 1x2 matrix for size (number of element) of row and size of column of A size(A,1) = 2 - size of row of A size(A,2) = 3 - size of column of A length (A) = 3 - size of longest dimension of A pwd - current direction/path currently in load filename.dat - load data into a variable called 'filename' who - list all variables in the current scope/workspace whos - similar to who but with details such as Size, Bytes, Class clear - delete al...

Computation Graph for Logistic Regression

Logistic Regression: \(z = w^T x + b\) \(\hat{y} = a = \sigma(z)\) \(L(a, y) = -(ylog(a) + (1-y)log(1-a))\) Then, computation graph is: \(\frac{\partial L}{\partial a} = -\frac{y}{a} + \frac{1-y}{1-a}\) and \(\frac{\partial a}{\partial z} = a(1-a) \) (derivative of sigmoid function) then, \(\frac{\partial L}{\partial z} = \frac{\partial L}{\partial a} * \frac{\partial a}{\partial z} = a-y\) Finally, \(\frac{\partial L}{\partial w_1} = \frac{\partial L}{\partial z} * \frac{\partial z}{\partial w_1} = (a-y) * x_1 \), \(\frac{\partial L}{\partial x_1} = \frac{\partial L}{\partial z} * \frac{\partial z}{\partial x_1} = (a-y) * w_1 \) \(\frac{\partial L}{\partial w_2} = \frac{\partial L}{\partial z} * \frac{\partial z}{\partial w_2} = (a-y) * x_2 \), \(\frac{\partial L}{\partial x_2} = \frac{\partial L}{\partial z} * \frac{\partial z}{\partial x_1} = (a-y) * w_2 \) \(\frac{\partial L}{\partial b} = \frac{\partial L}{\partial z} * \frac{\partial z}{\partial b} = (a-...

Powered by Blogger