CodeUtility
Making your developer's life easier!
AI Concepts in Python

Linear and Logistic Regression in Python

Regression is the simplest kind of machine learning. Linear regression predicts a number with a straight line; logistic regression passes that line through a sigmoid to predict the probability of a yes/no answer.

The concept in plain words

Regression is the simplest kind of machine learning. Linear regression predicts a number with a straight line; logistic regression passes that line through a sigmoid to predict the probability of a yes/no answer.

Fit a straight line with least squares, then turn a score into a probability with logistic regression.

Key ideas

  • Linear regression predicts continuous values such as prices or temperatures.
  • Logistic regression predicts a probability for two classes.
  • Both can be trained with gradient descent instead of a closed formula.
  • A perceptron is a related classifier that updates its weights only when it makes a mistake.

When should you use it?

  • Understand what happens inside neural networks and AI libraries.
  • Learn the idea before you implement it in the matching challenges.
  • Prepare for machine learning courses, interviews and projects.

See it in action

Click the left chart to add points and watch the best-fit line update. On the right, change w and b to move the decision boundary.

Example code

Run code →
main.py
import math

# Linear regression: find the line that best fits the points (least squares).
xs = [1, 2, 3, 4, 5]
ys = [2.1, 3.9, 6.2, 8.1, 9.8]
mean_x, mean_y = sum(xs) / len(xs), sum(ys) / len(ys)
slope = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys)) / sum((x - mean_x) ** 2 for x in xs)
intercept = mean_y - slope * mean_x
print(f"line: y = {slope:.2f}x + {intercept:.2f}")
print("predict x=6:", round(slope * 6 + intercept, 2))

# Logistic regression turns a score into a probability with the sigmoid.
def predict_pass(hours_studied, weight=1.5, bias=-4.0):
    return 1 / (1 + math.exp(-(weight * hours_studied + bias)))

for hours in (1, 3, 5):
    p = predict_pass(hours)
    print(f"{hours} hours -> pass probability {p:.2f} -> {'pass' if p >= 0.5 else 'fail'}")

Expected output

line: y = 1.96x + 0.14
predict x=6: 11.9
1 hours -> pass probability 0.08 -> fail
3 hours -> pass probability 0.62 -> pass
5 hours -> pass probability 0.97 -> pass

How it works

The least-squares formula finds the slope and intercept that make the squared distance to the points as small as possible, giving y = 1.96x + 0.14. predict_pass() computes a score from the hours studied and the sigmoid turns it into a probability; 0.5 is the cut-off between fail and pass.

Change the values and run the program in the CodeUtility online Python compiler without installing Python locally.

Practice exercises

Modify the runnable example with the exercises below to build understanding beyond copying the result.

  1. Change the inputs or weights and predict the output before running.
  2. Print the intermediate values to follow each step of the calculation.
  3. Solve the linked challenges below to implement the idea yourself.
Run in Python IDE →

Practice with a challenge

Use what this lesson covers to solve a real problem. Each challenge runs in your browser with instant tests and an AI review.

All challenges