Skip to content

Chapter 2 of 7

Inspecting, Selecting, and Filtering

Before manipulating a DataFrame you usually inspect it. df.head() and df.tail(n) peek at the first and last rows (default five). df.shape returns a tuple (n_rows, n_cols), df.columns lists the column labels, df.index exposes the row index, and df.dtypes reports the dtype of every column. For a single, human-readable summary, df.info() prints the shape, dtypes, non-null counts, and memory footprint in one shot.

Selecting data comes in two flavors: column access and row access. A single column is retrieved with df['col'] or the attribute form df.col, both of which return a one-dimensional Series. Multiple columns are requested by passing a list — df[['col1', 'col2']] — and produce a DataFrame. For rows, df.loc[row_labels, col_labels] does label-based selection while df.iloc[row_positions, col_positions] does integer-position-based selection; iloc[0] returns the first row, and iloc[5:10] slices rows five through nine (the end is exclusive). For fast scalar access, df.at['row', 'col'] and df.iat[2, 3] are quicker than loc and iloc respectively.

Filtering uses boolean masks. Equality with a value gives df[df['col'] == value], multiple AND-conditions are combined with &, and OR-conditions use |; each mask must be parenthesized so Python parses them correctly. Membership in a list is expressed with df[df['col'].isin(['a', 'b', 'c'])], missing values with df['col'].isna() (or .notna() for the inverse), and pattern matches with df['col'].str.contains('pattern', regex=True). As a shortcut, df.query('a > 0 and b < 10') lets you write the filter as a string expression that references column names directly without the df. prefix. When you need to assign to a filtered subset, prefer df.loc[mask, 'col'] = new_value over chained assignment like df[mask]['col'] = new_value, which can trigger a SettingWithCopyWarning and may silently fail.

All chapters
  1. 1Foundations and I/O
  2. 2Inspecting, Selecting, and Filtering
  3. 3Modifying and Sorting
  4. 4Aggregation and Apply
  5. 5Missing Data and Duplicates
  6. 6GroupBy, Combining, and Reshaping
  7. 7Time Series, Strings, and Categorical Types

Drill it

Reading is not remembering. These come from the Pandas Dataframe Operations deck:

Q

What is a Pandas DataFrame?

A two-dimensional, size-mutable, tabular data structure with labeled axes (rows and columns), similar to a spreadsheet or SQL table.

Q

What library provides DataFrame in Python?

Pandas (import pandas as pd).

Q

How do you create a DataFrame from a dictionary of lists?

pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})

Q

How do you create a DataFrame from a list of dictionaries?

pd.DataFrame([{'a': 1}, {'a': 2, 'b': 3}]); missing keys become NaN.