1 year ago

#363802

test-img

James Bell

Why is my decision boundary wrong for logistic regression using gradient descent?

I am trying to solve a classification task using logistic regression. Part of my task is to plot the decision boundary. I find that the gradient of the decision boundary seems to be solved correctly by my algorithm but when plotting the boundary is too high and does not separate the points well. I cannot work out why this is and would be grateful for any advice to solve this issue.

data = open('Question5.mat');
x = data.x; y = data.y; % Extract data for ease of use

LR = 0.001; % Set tunable learning rate for gradient descent

w_est = [0; 0; 0];  % Set inital guess for a, b, and c

cost = [];  % Initalise array to hold value of cost function
figure;
for i = 1:20000     % Set iteration limit for gradient descent

    iteration_cost = 0; grad_a = 0; grad_b = 0; grad_c = 0;     % Set innial value of 0 for summed terms
    
    for m = 1:1100  % Iterate through data points
        y_hat_est = 1./(1+exp(-w_est'*[x(m,1); x(m,2); 1]));    % Calculate value of sigmoid function with estimated coefficients for each datapoint
        iteration_cost = iteration_cost + y(m)*log(y_hat_est)+(1-y(m))*log(1-y_hat_est);    % Calculate cost function and add it to summed term for each data point
        
        % Calculate each gradient term for each data point and add to
        % summed gradient
        grad_a = grad_a + (y_hat_est - y(m))*x(m,1);
        grad_b = grad_b + (y_hat_est - y(m))*x(m,2);
        grad_c = grad_c + (y_hat_est - y(m))*x(m,3);
    
    end
    
    
    g = [grad_a; grad_b; grad_c];   % Create vector of gradients
    
    w_est = w_est - LR*g;   % Update estimate vector with next term
    cost(i) = -iteration_cost;  % Add the value of the cost function to the array for costs
    
    if mod(i,1000) == 0    % Only plot on some iterations to speed up program
        hold off
        gscatter(x(:,1),x(:,2),y,'rb'); % Plot scatter plot grouped by class
        xlabel('x1'); ylabel('x2'); title(i);   % Add title and labels to figure
        hold on
        x1_plot = -6:4; x2_plot = -3:7; % Create array of values for plotting
        plot( -(w_est(1)*x1_plot + w_est(3)) /w_est(2), x2_plot);   % Plot decision boundary based on the current coefficient estimates

%         pause(1)  % Add delay to aid visualisation
end
end
hold off;
figure; plot(cost)  % Plot the cost function
title('Cost function'); xlabel('Iteration number'); ylabel('cost');

enter image description here enter image description hereenter image description here

matlab

machine-learning

logistic-regression

gradient-descent

0 Answers

Your Answer

Accepted video resources