Skip to content

Chapter 7 of 7

Data Inspection and Jupyter Productivity

Before any serious analysis, you need a quick overview of a DataFrame. df.shape returns a tuple of (rows, columns), df.head(10) shows the first ten rows, and df.dtypes lists the data type of each column. The methods df.info() and df.describe() are particularly useful: info() prints column names, non-null counts, dtypes, and memory usage, giving a one-glance health check of the data, while describe() returns summary statistics such as count, mean, standard deviation, and the 25%, 50%, and 75% percentiles for every numeric column.

For categorical or discrete columns, df['col'].value_counts() returns the frequency of each unique value sorted in descending order, while df['col'].unique() lists the distinct values and df['col'].nunique() returns their count. The correlation matrix df.corr() produces pairwise Pearson correlation coefficients for all numeric columns and is often the first step toward understanding linear relationships between features.

Jupyter notebooks provide a rich environment for iterative data science, and a few built-in tools make them even more productive. The line magic %timeit runs a statement many times and reports its average execution time, which is ideal for benchmarking small snippets, while the cell magic %%time measures the wall time and CPU time of an entire cell in a single run. Keyboard shortcuts speed up navigation: Shift + Enter runs the current cell and advances, pressing Esc enters command mode, and A inserts a new cell above the active one. Finally, data flows in and out of notebooks through simple I/O functions: pd.read_csv('file.csv') loads a CSV (or a TSV with sep='\t'), and df.to_csv('output.csv', index=False) writes a DataFrame to disk, with index=False preventing the index from being saved as an extra column.

All chapters
  1. 1NumPy Essentials
  2. 2Pandas DataFrames and Selection
  3. 3Data Cleaning and Type Handling
  4. 4Data Visualization with Matplotlib and Seaborn
  5. 5Aggregation, Merging, and Reshaping
  6. 6Feature Engineering and Preprocessing
  7. 7Data Inspection and Jupyter Productivity

Drill it

Reading is not remembering. These come from the Python For Data Science deck:

Q

What function creates a NumPy array from a Python list?

np.array([1, 2, 3]) converts a Python list into a NumPy ndarray.

Q

How do you create a 1D array of zeros with 5 elements in NumPy?

np.zeros(5) returns array([0., 0., 0., 0., 0.]).

Q

How do you create a 3x3 identity matrix in NumPy?

np.eye(3) returns a 3×3 array with ones on the diagonal and zeros elsewhere.

Q

What does <code>np.arange(0, 10, 2)</code> return?

It returns array([0, 2, 4, 6, 8]) — values from 0 up to (not including) 10 with step 2.