Learning Objectives – after this chapter you will be able to:
- Explain the CSV format.
- Read CSV files into a DataFrame with
pd.read_csv()and its common options. - Explore data with
head(),tail(),info()anddescribe(). - Handle missing values and save results with
to_csv().
What is a CSV File?
- CSV (Comma-Separated Values) is a plain-text format for tables. Each line is a row, and values in a row are separated by commas. The first line usually contains the column names (header).
- CSV files can be opened in Excel, Google Sheets and any text editor, which makes them a common way to exchange data.
Roll,Name,Course,Marks 1,Asha,BCA,82 2,Ravi,BBA,67 3,Meera,BCA, 4,John,MCA,74 5,Sara,BBA,58
Meera’s marks are empty – a missing value, which pandas reads as NaN.
Reading a CSV File: read_csv()
import pandas as pd
# Create a small CSV file for practice
csv_text = """Roll,Name,Course,Marks
1,Asha,BCA,82
2,Ravi,BBA,67
3,Meera,BCA,
4,John,MCA,74
5,Sara,BBA,58
"""
with open("students.csv", "w") as f:
f.write(csv_text)
df = pd.read_csv("students.csv")
print(df)
print(df.shape)Output
Roll Name Course Marks 0 1 Asha BCA 82.0 1 2 Ravi BBA 67.0 2 3 Meera BCA NaN 3 4 John MCA 74.0 4 5 Sara BBA 58.0 (5, 4)
Marks are shown as 82.0, 67.0 … because the column contains a missing value (
NaN), which is a float, so pandas stores the whole column as float.Common read_csv() options
| Option | Purpose |
|---|---|
sep=";" | Use a different separator (e.g. semicolon or "\t" for tab). |
index_col="Roll" | Use a column as the row index. |
usecols=["Name", "Marks"] | Read only some columns. |
nrows=100 | Read only the first 100 rows. |
header=None, names=[...] | File has no header; supply column names. |
na_values=["-", "NA"] | Extra strings to treat as missing. |
encoding="utf-8" | Text encoding of the file. |
Using options
import pandas as pd
with open("students.csv", "w") as f:
f.write("Roll,Name,Course,Marks\n1,Asha,BCA,82\n2,Ravi,BBA,67\n3,Meera,BCA,\n4,John,MCA,74\n5,Sara,BBA,58\n")
small = pd.read_csv("students.csv", usecols=["Name", "Marks"], nrows=2)
print(small)
by_roll = pd.read_csv("students.csv", index_col="Roll")
print(by_roll.loc[4, "Name"])Output
Name Marks 0 Asha 82 1 Ravi 67 John
Reading a CSV in Google Colab (upload or Google Drive)
import pandas as pd
from google.colab import files
uploaded = files.upload() # choose a CSV from your computer
df = pd.read_csv("students.csv")
# or read from Google Drive after drive.mount('/content/drive')
df = pd.read_csv("/content/drive/MyDrive/students.csv")Exploring the Data
| Method / attribute | Shows |
|---|---|
head(n) / tail(n) | First / last n rows (default 5). |
shape, columns, dtypes | Size, column names and data types. |
info() | Columns, non-null counts, data types and memory use. |
describe() | Statistics of numeric columns: count, mean, std, min, quartiles, max. |
import pandas as pd
with open("students.csv", "w") as f:
f.write("Roll,Name,Course,Marks\n1,Asha,BCA,82\n2,Ravi,BBA,67\n3,Meera,BCA,\n4,John,MCA,74\n5,Sara,BBA,58\n")
df = pd.read_csv("students.csv")
print(df.head(2))
print(df.tail(1))
print(df["Marks"].describe())Output
Roll Name Course Marks 0 1 Asha BCA 82.0 1 2 Ravi BBA 67.0 Roll Name Course Marks 4 5 Sara BBA 58.0 count 4.000000 mean 70.250000 std 10.210289 min 58.000000 25% 64.750000 50% 70.500000 75% 76.000000 max 82.000000 Name: Marks, dtype: float64
info() – prints a summary of every column (output depends on your pandas version)
df.info()
Handling Missing Values
import pandas as pd
with open("students.csv", "w") as f:
f.write("Roll,Name,Course,Marks\n1,Asha,BCA,82\n2,Ravi,BBA,67\n3,Meera,BCA,\n4,John,MCA,74\n5,Sara,BBA,58\n")
df = pd.read_csv("students.csv")
print(df.isna().sum().tolist()) # missing values per column
print(len(df.dropna())) # rows left after dropping
filled = df.fillna({"Marks": df["Marks"].mean()})
print(filled["Marks"].tolist())Output
[0, 0, 0, 1] 4 [82.0, 67.0, 70.25, 74.0, 58.0]
Saving a DataFrame: to_csv()
Use index=False so that the row numbers are not written as an extra column.
import pandas as pd
with open("students.csv", "w") as f:
f.write("Roll,Name,Course,Marks\n1,Asha,BCA,82\n2,Ravi,BBA,67\n3,Meera,BCA,\n4,John,MCA,74\n5,Sara,BBA,58\n")
df = pd.read_csv("students.csv")
toppers = df[df["Marks"] >= 70]
toppers.to_csv("toppers.csv", index=False)
with open("toppers.csv") as f:
print(f.read())Output
Roll,Name,Course,Marks 1,Asha,BCA,82.0 4,John,MCA,74.0
Key Takeaways
pd.read_csv()loads a CSV into a DataFrame; options control separator, columns, rows and index.- Explore with
head(),info()anddescribe(); empty cells become NaN. - Clean with
dropna()/fillna()and save withto_csv(index=False).
Video : CSV Files and DataFrames
Question / Answer (15 questions)
Answer : A CSV (Comma-Separated Values) file is a plain-text file that stores a table: each line is a row and the values in a row are separated by commas. The first line usually holds the column names.
Answer : Use pd.read_csv("file.csv"), which returns a DataFrame. The first row is used as the header by default.
Answer : Pass the separator: pd.read_csv("file.csv", sep=";") or sep="\t" for tab-separated files.
Answer : Use the index_col option, e.g. pd.read_csv("students.csv", index_col="Roll").
Answer : Use usecols=["Name", "Marks"] to select columns and nrows=100 to read only the first 100 rows.
Answer : Use header=None and provide column names with names=[...], e.g. pd.read_csv("data.csv", header=None, names=["Roll", "Name"]).
Answer : head(n) returns the first n rows and tail(n) the last n rows of a DataFrame; n is 5 by default. They are used for a quick look at the data.
Answer : info() prints the number of rows, each column’s name, non-null count and data type, and the memory usage of the DataFrame.
Answer : For numeric columns it shows count, mean, standard deviation, minimum, 25th/50th/75th percentiles and maximum.
Answer : As NaN (Not a Number), the marker for a missing value. A numeric column containing NaN is stored as float.
Answer : df.isna().sum() returns the number of missing values per column.
Answer : dropna() removes rows (or columns) containing missing values. fillna(value) replaces missing values with a given value, such as 0 or the column mean.
Answer : df.fillna({"Marks": df["Marks"].mean()}) or df["Marks"] = df["Marks"].fillna(df["Marks"].mean()).
Answer : df.to_csv("output.csv", index=False). Without index=False the row index is written as an extra unnamed first column.
Answer : Mount Drive with from google.colab import drive; drive.mount('/content/drive'), then read it with pd.read_csv("/content/drive/MyDrive/students.csv").
MCQ Test : Check Your Understanding (5 questions)
Select an option and press Check Answer, or answer all questions and press Submit Test to see your score. Each answer comes with an explanation.
Bibliography/References
The notes, questions and MCQs on this page are original explanations written for this course. The works listed below were consulted and are cited for attribution and further reading. Product names, logos and trademarks belong to their respective owners. Short quotations, where used, are marked and attributed.
- pandas development team. (n.d.). IO tools (text, CSV, HDF5, …). pandas User Guide. https://pandas.pydata.org/docs/user_guide/io.html
- pandas development team. (n.d.). Working with missing data. pandas User Guide. https://pandas.pydata.org/docs/user_guide/missing_data.html
- pandas development team. (n.d.). 10 minutes to pandas. pandas User Guide. https://pandas.pydata.org/docs/user_guide/10min.html
- pandas development team. (n.d.). Getting started tutorials. pandas Documentation. https://pandas.pydata.org/docs/getting_started/intro_tutorials/index.html
- Python Software Foundation. (n.d.). csv — CSV File Reading and Writing. The Python Standard Library. https://docs.python.org/3/library/csv.html
- Google Colaboratory. (n.d.). External data: Local Files, Drive, Sheets, and Cloud Storage (example notebook). https://colab.research.google.com/notebooks/io.ipynb
- McKinney, W. (2022). Python for Data Analysis (3rd ed.). O’Reilly Media. (Open-access online edition.) https://wesmckinney.com/book/