Then we rotate the line a little bit and do the same. The line with the smallest sum of squared residuals is the line chosen to fit best.
Why can’t we make use of least-squares to find the best fitting line in logistic regression?
Well, to answer this we need to recall logistic regression. Our goal in logistic regression is to draw the best fitting S-curve for given data points. And in logistic regression, we transform the y-axis from the probabilities to log(odds). The problem is that this transformation pushes the data points to positive and negative infinity as shown below