Learning Objectives – after this chapter you will be able to:
- Explain what pandas is and what it is used for.
- Install, import and check the version of pandas.
- Describe the two main data structures: Series and DataFrame.
- Outline a typical data-analysis workflow with pandas.
What is Pandas?
- pandas is an open-source Python library for data manipulation and analysis. It makes it easy to load, clean, transform, analyse and save tabular data – the kind of data found in spreadsheets, CSV files and databases.
- It was created by Wes McKinney (development began in 2008). The name comes from “panel data”, an econometrics term for multi-dimensional data sets.
- pandas is built on top of NumPy, so it is fast and works well with NumPy, Matplotlib and scikit-learn.
What can pandas do?
- Read and write data in many formats: CSV, Excel, JSON, SQL databases and more.
- Clean data: handle missing values, remove duplicates, fix data types.
- Select, filter, sort, group, merge and reshape data.
- Calculate statistics such as mean, median, correlation and totals.
Installing and Importing
- pandas is pre-installed in Google Colab. On your own computer install it with
pip install pandas. - It is conventionally imported as
pd.
Import pandas and check the version
import pandas as pd print(pd.__version__) # shows the installed version, e.g. 2.2.2
Pandas Data Structures
| Structure | Dimensions | Description |
|---|---|---|
| Series | 1-D | A labelled column of values – like one column of a spreadsheet. Every value has an index label. |
| DataFrame | 2-D | A table of rows and columns. Each column is a Series; columns can have different data types. |
A Series
import pandas as pd marks = pd.Series([82, 67, 91], index=["Asha", "Ravi", "Meera"]) print(marks) print(marks["Meera"])
Output
Asha 82 Ravi 67 Meera 91 dtype: int64 91
A DataFrame
import pandas as pd
df = pd.DataFrame({
"Name": ["Asha", "Ravi", "Meera"],
"Age": [20, 21, 19],
"Marks": [82, 67, 91],
})
print(df)
print(df.shape)
print(df["Marks"].mean())Output
Name Age Marks 0 Asha 20 82 1 Ravi 21 67 2 Meera 19 91 (3, 3) 80.0
Pandas vs NumPy
| Basis | NumPy | pandas |
|---|---|---|
| Main object | ndarray (N-dimensional) | Series (1-D) and DataFrame (2-D) |
| Data types | One type per array | Each column can have its own type |
| Labels | Positions only | Row index labels and column names |
| Best for | Numerical computation | Tabular data analysis, cleaning and file I/O |
Typical Workflow
1. Load
pd.read_csv()2. Explore
head(), info(), describe()3. Cleanmissing values, duplicates, types
4. Analysefilter, sort, group, statistics
5. Save
to_csv()Key Takeaways
- pandas is the standard Python library for tabular data analysis, built on NumPy.
- Import it as
pd; it is pre-installed in Colab. - Series = labelled 1-D column; DataFrame = 2-D table of Series.
Video : Pandas Basics
Question / Answer (15 questions)
Answer : pandas is an open-source Python library for data manipulation and analysis. It provides fast, flexible data structures – Series and DataFrame – for working with labelled, tabular data.
Answer : pandas was created by Wes McKinney, who began developing it in 2008. The name is derived from “panel data”, an econometrics term for multi-dimensional structured data sets.
Answer : Install it with pip install pandas (it is already available in Google Colab) and import it with import pandas as pd.
Answer : Print pd.__version__ after importing pandas.
Answer : Series, a one-dimensional labelled array, and DataFrame, a two-dimensional labelled table of rows and columns.
Answer : A Series is a one-dimensional labelled array that can hold data of any type. Each value is associated with an index label, like one column of a spreadsheet.
Answer : A DataFrame is a two-dimensional labelled data structure with rows and columns, similar to a spreadsheet or SQL table. Each column is a Series and columns can have different data types.
Answer : A DataFrame is a collection of Series that share the same row index; selecting one column of a DataFrame returns a Series.
Answer : Reading and writing files (CSV, Excel, JSON, SQL), cleaning data (missing values, duplicates, types), selecting/filtering/sorting data, and grouping and computing statistics.
Answer : pandas is built on NumPy; its columns are stored using NumPy arrays (or similar array types), which gives it fast numerical performance.
Answer : A NumPy array holds a single data type and is indexed by position, while a pandas DataFrame can hold a different data type in each column and has row labels and column names. NumPy is aimed at numerical computing; pandas is aimed at tabular data analysis.
Answer : A tuple (number of rows, number of columns). For a DataFrame of 3 students and 3 columns it returns (3, 3).
Answer : Load the data (read_csv), explore it (head, info, describe), clean it (missing values, duplicates, types), analyse it (filter, sort, group, statistics) and save the results (to_csv).
Answer : A RangeIndex of integers starting from 0 (0, 1, 2, …), unless an index is specified.
Answer : Use its label: marks["Meera"] returns 91 for marks = pd.Series([82, 67, 91], index=["Asha", "Ravi", "Meera"]).
MCQ Test : Check Your Understanding (5 questions)
Select an option and press Check Answer, or answer all questions and press Submit Test to see your score. Each answer comes with an explanation.
Bibliography/References
The notes, questions and MCQs on this page are original explanations written for this course. The works listed below were consulted and are cited for attribution and further reading. Product names, logos and trademarks belong to their respective owners. Short quotations, where used, are marked and attributed.
- pandas development team. (n.d.). 10 minutes to pandas. pandas User Guide. https://pandas.pydata.org/docs/user_guide/10min.html
- pandas development team. (n.d.). Intro to data structures. pandas User Guide. https://pandas.pydata.org/docs/user_guide/dsintro.html
- pandas development team. (n.d.). Installation. pandas Documentation. https://pandas.pydata.org/docs/getting_started/install.html
- pandas development team. (n.d.). Getting started tutorials. pandas Documentation. https://pandas.pydata.org/docs/getting_started/intro_tutorials/index.html
- McKinney, W. (2010). Data Structures for Statistical Computing in Python. In Proceedings of the 9th Python in Science Conference (pp. 56–61). https://doi.org/10.25080/Majora-92bf1922-00a
- McKinney, W. (2022). Python for Data Analysis (3rd ed.). O’Reilly Media. (Open-access online edition.) https://wesmckinney.com/book/