Learning Objectives – after this chapter you will be able to:
- Filter arrays with a boolean index list.
- Create filters directly from conditions.
- Combine conditions with
&,|and~. - Replace or count values using conditions.
What is Filtering?
Filtering means taking some elements out of an existing array to create a new array. In NumPy you filter with a boolean index list: an array of True/False values of the same length. Elements at True positions are kept.
import numpy as np arr = np.array([41, 42, 43, 44]) keep = [True, False, True, False] print(arr[keep])
Output
[41 43]
Creating a Filter from a Condition
Writing the boolean list by hand is rarely needed. A condition applied to the array produces the filter automatically.
import numpy as np
arr = np.array([12, 45, 7, 30, 18, 51])
# Step-by-step with a loop (to understand the idea)
manual = []
for x in arr:
manual.append(x > 20)
print(manual == list(arr > 20))
# The NumPy way
mask = arr > 20
print(mask)
print(arr[mask])
print(arr[arr % 2 == 0]) # even numbersOutput
True [False True False True False True] [45 30 51] [12 30 18]
Combining Conditions
Use & (and), | (or) and ~ (not). Each condition must be in parentheses, because these operators bind more tightly than comparisons. Python’s and/or do not work on arrays.
import numpy as np
marks = np.array([35, 78, 92, 55, 61, 88])
print(marks[(marks >= 60) & (marks < 90)])
print(marks[(marks < 40) | (marks > 90)])
print(marks[~(marks >= 60)])
try:
marks[marks > 50 and marks < 80]
except ValueError:
print("Use & and | (not 'and'/'or') with arrays")Output
[78 61 88] [35 92] [35 55] Use & and | (not 'and'/'or') with arrays
Filtering 2-D Arrays and Rows
import numpy as np m = np.array([[5, 12, 8], [20, 3, 15]]) print(m[m > 10]) # result is 1-D data = np.array([[101, 45], [102, 82], [103, 67]]) # [id, marks] print(data[data[:, 1] >= 60]) # rows where marks >= 60
Output
[12 20 15] [[102 82] [103 67]]
Replacing and Counting with Conditions
import numpy as np
temps = np.array([31, -999, 29, 35, -999, 33]) # -999 = missing reading
print(np.where(temps == -999, 0, temps)) # replace (returns new array)
print(np.where(temps > 30, "Hot", "OK"))
valid = temps[temps != -999]
print(valid, valid.mean())
print("Missing readings:", np.count_nonzero(temps == -999))
temps[temps == -999] = 30 # replace in place
print(temps)Output
[31 0 29 35 0 33] ['Hot' 'OK' 'OK' 'Hot' 'OK' 'Hot'] [31 29 35 33] 32.0 Missing readings: 2 [31 30 29 35 30 33]
Key Takeaways
- A boolean mask keeps elements at True positions;
arr[arr > x]is the usual form. - Combine conditions with
&,|,~and put each condition in parentheses. np.where(cond, a, b)replaces values; masks can also count and assign.
Video : Array Filter
Question / Answer (15 questions)
Answer : Filtering is creating a new array from only those elements of an existing array that meet a condition.
Answer : A list or array of True/False values with the same length as the array. When used as an index, elements at True positions are kept and those at False positions are dropped.
Answer : [41 43] – only the elements at the True positions are selected.
Answer : Apply a comparison to the array, e.g. mask = arr > 20, which returns a boolean array. Then use it as an index: arr[mask] or simply arr[arr > 20].
Answer : arr[arr % 2 == 0].
Answer : & for AND, | for OR and ~ for NOT. Each condition must be enclosed in parentheses, e.g. arr[(arr >= 60) & (arr < 90)].
Answer : and and or need a single True/False value, but an array of many booleans has no single truth value, so Python raises ValueError ("The truth value of an array with more than one element is ambiguous").
Answer : & and | have higher precedence than comparison operators. Without parentheses, arr > 50 & arr < 80 would be evaluated as arr > (50 & arr) < 80, which is wrong or raises an error.
Answer : The result is a 1-D array of the selected elements, because the kept elements no longer form a rectangle.
Answer : data[data[:, 1] >= 60]. The 1-D mask on the column selects whole rows.
Answer : It returns a new array taking elements from x where the condition is True and from y where it is False, e.g. np.where(temps == -999, 0, temps) replaces -999 with 0.
Answer : Assign to the masked positions: arr[arr == -999] = 30 changes those elements directly in the original array.
Answer : np.count_nonzero(condition) or condition.sum(), e.g. np.count_nonzero(temps == -999).
Answer : A copy. Changing the filtered result does not change the original array (assignment through arr[mask] = value does change it, because it modifies the original directly).
Answer : Filter first, then average: temps[temps != -999].mean().
MCQ Test : Check Your Understanding (5 questions)
Select an option and press Check Answer, or answer all questions and press Submit Test to see your score. Each answer comes with an explanation.
Bibliography/References
The notes, questions and MCQs on this page are original explanations written for this course. The works listed below were consulted and are cited for attribution and further reading. Product names, logos and trademarks belong to their respective owners. Short quotations, where used, are marked and attributed.
- NumPy Developers. (n.d.). Indexing on ndarrays. NumPy User Guide. https://numpy.org/doc/stable/user/basics.indexing.html
- NumPy Developers. (n.d.). numpy.where. NumPy API Reference. https://numpy.org/doc/stable/reference/generated/numpy.where.html
- NumPy Developers. (n.d.). NumPy quickstart. NumPy User Guide. https://numpy.org/doc/stable/user/quickstart.html
- NumPy Developers. (n.d.). NumPy: the absolute basics for beginners. NumPy User Guide. https://numpy.org/doc/stable/user/absolute_beginners.html
- VanderPlas, J. (2016). Python Data Science Handbook. O’Reilly Media. (Free online edition.) https://jakevdp.github.io/PythonDataScienceHandbook/