AI Concepts in Python

Forward Propagation and Loss Functions in Python

A neural network is layers of neurons. In a forward pass the input flows through each layer to produce a prediction, and a loss function scores how far that prediction is from the right answer.

The concept in plain words

A neural network is layers of neurons. In a forward pass the input flows through each layer to produce a prediction, and a loss function scores how far that prediction is from the right answer.

Pass inputs through a small neural network layer by layer, then measure the error with squared error and cross-entropy.

Key ideas

A dense layer connects every input to every neuron in the layer.
The output of one layer is the input of the next one.
The loss is a single number, and lower means better predictions.
Use squared error for numbers and cross-entropy for probabilities and classes.

When should you use it?

Understand what happens inside neural networks and AI libraries.
Learn the idea before you implement it in the matching challenges.
Prepare for machine learning courses, interviews and projects.

See it in action

Change the inputs and the target. Follow the values through the hidden layer and see how both losses react to the prediction.

Example code

Run code →

main.py

import math

def dense(inputs, weights, biases, activation):
    # Each row of weights belongs to one neuron in the layer.
    return [activation(sum(x * w for x, w in zip(inputs, row)) + b) for row, b in zip(weights, biases)]

relu = lambda z: max(0.0, z)
sigmoid = lambda z: 1 / (1 + math.exp(-z))

x = [1.0, 2.0]
hidden = dense(x, [[0.5, -0.2], [0.3, 0.8]], [0.0, -0.5], relu)
output = dense(hidden, [[1.0, -1.0]], [0.2], sigmoid)
print("hidden layer:", [round(h, 3) for h in hidden])
print("prediction:", round(output[0], 3))

# A loss function measures how wrong the prediction is.
target = 1.0
mse = (output[0] - target) ** 2
bce = -(target * math.log(output[0]) + (1 - target) * math.log(1 - output[0]))
print("squared error:", round(mse, 3))
print("cross-entropy:", round(bce, 3))

Expected output

hidden layer: [0.1, 1.4]
prediction: 0.25
squared error: 0.563
cross-entropy: 1.387

How it works

dense() computes one layer, where every row of weights is one neuron. The hidden layer uses ReLU and the output layer uses sigmoid to give a probability. Squared error and cross-entropy both measure the gap between the prediction 0.25 and the target 1.0; training tries to make them smaller.

Change the values and run the program in the CodeUtility online Python compiler without installing Python locally.

Practice exercises

Modify the runnable example with the exercises below to build understanding beyond copying the result.

Change the inputs or weights and predict the output before running.
Print the intermediate values to follow each step of the calculation.
Solve the linked challenges below to implement the idea yourself.

Run in Python IDE →

Practice with a challenge

Use what this lesson covers to solve a real problem. Each challenge runs in your browser with instant tests and an AI review.

All challenges