Handle Missing Values in pandas
Find empty cells, then fill them or drop them so your calculations do not break.
- Finding NaN with isna()
- fillna() with a value or the median
- dropna() to remove rows
Ada95LinNaN → 90.0 (median)Sam85Find, then fill or drop
The problem
Real data has gaps: an unanswered survey question, a sensor that missed a reading, a cell left empty. Many calculations break or give misleading answers if the gaps are ignored, so you have to find them and decide what to do.
Think of it like this
Handling missing values is like a teacher whose gradebook has an empty cell. They can leave the student out, or fill the cell with a fair estimate such as the middle grade of the class. Either way the decision is made on purpose.
np.nan⬜ The empty cellstudents["score"].median()📏 A fair estimate: the middle score, 90.fillna(...)🖊️ Fill the empty cell with itstudents["score"] = ...💾 Save the filled column backMissing values are cells with no data, shown as NaN; you can fill them with a value such as the median or remove the rows that contain them.
Key points
np.nanmarks Lin's score as missing.students["score"].median()ignores the gap and returns 90.0, the middle of 95 and 85..fillna(...)replaces the gap with that value.- Assigning back to
students["score"]keeps the change; the column becomes float because NaN is a float.
When should you use it?
- Cleaning survey, sensor and form data before analysis.
- Filling gaps with a typical value so calculations can run.
- Counting gaps with
isna().sum()to judge data quality.
When not to use it
- Most values in a column are missing: filling them invents data; consider dropping the column.
- The fact that a value is missing is itself important: keep a separate True/False column before filling.
Example code
import pandas as pd
import numpy as np
students = pd.DataFrame({"name": ["Ada", "Lin", "Sam"], "score": [95, np.nan, 85]})
students["score"] = students["score"].fillna(students["score"].median())
print(students)
Result of running the code
name score
0 Ada 95.0
1 Lin 90.0
2 Sam 85.0
Code explanation
The median is calculated from available scores and used to replace NaN. Assigning back to the column makes the change explicit.
Change the values and run the program in the CodeUtility online Python compiler without installing Python locally.
Practice exercises
Modify the runnable example with the exercises below to build understanding beyond copying the result.
- Add rows, columns, and missing values.
- Combine filtering, sorting, and groupby into a small report.
- Load a real file, inspect dtypes, and handle invalid data.
Common mistakes
Not saving the result
fillna returns a new Series; without assigning it, nothing changes.
students["score"].fillna(students["score"].median())
students["score"] = students["score"].fillna(students["score"].median())
Finished this lesson?
Mark it as learned to track your progress.