Skip to main content

Machine Learning

Search This Blog

Backpropagation Algorithm 2

Get link
Facebook
X
Pinterest
Email
Other Apps

Backpropagation Algorithm

$g'(z^{(3)})$ = derivative of activation function g evaluate at

$z^{(3)}$

Resources:
- https://en.wikibooks.org/wiki/Artificial_Neural_Networks/Activation_Functions
- https://www.coursera.org/learn/machine-learning/supplement/pjdBA/backpropagation-algorithm
- https://www.coursera.org/learn/machine-learning/supplement/v5Bu8/backpropagation-intuition

Backpropagation Algorithm

Get link
Facebook
X
Pinterest
Email
Other Apps

Comments

Post a Comment

Popular Posts

Octave Cheatsheet

Octave Commands: Arithmetics: +, -, *, /, ^ Logical operations: ==, ~= (not equal), &&, ||, xor Change prompt: PS1('>> '); Semicolon: suppressing output A = [1 2; 3 4; 5 6] - a 3x2 matrix B = [1; 2; 3; 4; 5] - a 5x1 matrix v = 1:0.1:2 - row matrix from 1 to 2 with 0.1 increment ones(2, 3) - 2x3 matrix of all 1s zeros(1,3) - 1x3 matrix of all 0s w = rand(1,3) - 1x3 matrix of all random numbers hist(w) - histogram eye (4) - 4x4 identity matrix help eye - how to use 'eye' function size(A) = [2, 3] - 1x2 matrix for size (number of element) of row and size of column of A size(A,1) = 2 - size of row of A size(A,2) = 3 - size of column of A length (A) = 3 - size of longest dimension of A pwd - current direction/path currently in load filename.dat - load data into a variable called 'filename' who - list all variables in the current scope/workspace whos - similar to who but with details such as Size, Bytes, Class clear - delete al...

Multiclass Classification: One-vs-all

Multiclass Classification: One-vs-all Classify more than just 0 and 1. Examples: Email foldering/tagging: Work (y=1), Friends (2), Family (3), Hobby (4) Medical diagrams: Not ill (1), Cold (2), Flu (3) Weather: Sunny (1), Cloudy (2), Rain (3), Snow (4) Summary by Filip: Logistic regression is a classification algorithm and not a regression algorithm, although "regression" is part of the name Hypothesis equals sigmoid/logistic function of theta_transpose * x The derivative of J(theta) is the same as for linear regression, i.e. 1/m * sum((h(x) - y)*x). However, the hypothesis is different. While the hypothesis in linear regression is h(x) = theta_transpose * X , the hypothesis in logistic regression is 1/(1+e^(theta_transpose * x)) Function z does not have to be linear; it can be everything, e.g. circle When dealing with multiclass logistic regression, if we follow the one vs. all approach, we can simplify the problem to simple binary classification problem....

Feature Scaling and Mean Normalization

Feature Scaling Idea: Make sure features are on a similar scale Example: below data, the contour will be very skewed hence gradient descent might take a long time to complete. We should convert them into similar scale, ie. similar range of values (in picture below they're now in the range of [0, 1] (if possible in the range of [-1, 1] )

$x_1$ = size (0-2000 feet 2 )

$x_2$ = number of bedrooms (1-5) Goal: get every feature into approximately [-1, 1] or [-0.5, 0.5] range Note: denominator is actually a range (= max - min), ie. 2000 - 0 = 2000 Mean normalization: Idea: replace

$x_i$ with

$x_i - \mu_i$ where

$\mu_i$ is the average of all the values for feature,

$s_i$ is the range of values (max - min) or

$s_i$ is the standard deviation. Formula:

$x_i := \frac{x_i - \mu_i}{s_i}$ Resources: - https://www.coursera.org/learn/machine-learning/lecture/xx3Da/gradient-descent-in-practice-i-feature-scaling

Powered by Blogger