Skip to content

Chapter 2 of 7

Pandas DataFrames and Selection

Pandas builds on NumPy to provide labeled, tabular data structures ideal for data analysis. The primary object is the DataFrame, which can be created directly from a dictionary such as pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]}). Once you have a DataFrame, the two main ways of selecting data are df.loc[] and df.iloc[]. The loc accessor is label-based, using the actual index values and column names, whereas iloc is purely integer-position based. For example, df.loc[0:3, ['name', 'age']] returns rows 0 through 3 together with only the 'name' and 'age' columns, while df.iloc[:5, :2] selects the first five rows and first two columns by position.

Boolean indexing is the standard way to filter rows in Pandas. Writing df[df['age'] > 30] yields a DataFrame containing only the rows where the condition holds. Multiple conditions can be combined by wrapping each one in parentheses and joining them with & (and) or | (or), as in df[(df['age'] > 25) & (df['city'] == 'NYC')]. Sorting is straightforward with df.sort_values('price', ascending=False), and you can sort by multiple columns with different orders using df.sort_values(['col1', 'col2'], ascending=[True, False]).

DataFrame indices are flexible and can be modified as needed. df.set_index('col') promotes a column to the row index, df.reset_index(drop=True) restores the default integer index while discarding the old one, and df.rename(columns={'old_name': 'new_name'}) renames selected columns. You can also create new columns directly through vectorized arithmetic, such as df['total'] = df['price'] * df['quantity'], or by applying a custom function element-wise with df['col'].apply(lambda x: x ** 2). These operations together make Pandas the most natural tool for the interactive slicing and dicing of tabular data.

All chapters
  1. 1NumPy Essentials
  2. 2Pandas DataFrames and Selection
  3. 3Data Cleaning and Type Handling
  4. 4Data Visualization with Matplotlib and Seaborn
  5. 5Aggregation, Merging, and Reshaping
  6. 6Feature Engineering and Preprocessing
  7. 7Data Inspection and Jupyter Productivity

Drill it

Reading is not remembering. These come from the Python For Data Science deck:

Q

What function creates a NumPy array from a Python list?

np.array([1, 2, 3]) converts a Python list into a NumPy ndarray.

Q

How do you create a 1D array of zeros with 5 elements in NumPy?

np.zeros(5) returns array([0., 0., 0., 0., 0.]).

Q

How do you create a 3x3 identity matrix in NumPy?

np.eye(3) returns a 3×3 array with ones on the diagonal and zeros elsewhere.

Q

What does <code>np.arange(0, 10, 2)</code> return?

It returns array([0, 2, 4, 6, 8]) — values from 0 up to (not including) 10 with step 2.