Title: Pandas Dataframe Operations — LearnCoachAssist

Q: What is a Pandas DataFrame?
A: A two-dimensional, size-mutable, tabular data structure with labeled axes (rows and columns), similar to a spreadsheet or SQL table.

Q: What library provides DataFrame in Python?
A: Pandas (import pandas as pd).

Q: How do you create a DataFrame from a dictionary of lists?
A: pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})

Q: How do you create a DataFrame from a list of dictionaries?
A: pd.DataFrame([{'a': 1}, {'a': 2, 'b': 3}]); missing keys become NaN.

Q: How do you read a CSV file into a DataFrame?
A: pd.read_csv('file.csv').

Q: How do you read an Excel file into a DataFrame?
A: pd.read_excel('file.xlsx', sheet_name='Sheet1').

Q: How do you read a JSON file into a DataFrame?
A: pd.read_json('file.json') or pd.read_json('file.json', orient='records') for line-delimited JSON.

Q: How do you write a DataFrame to a CSV file?
A: df.to_csv('file.csv', index=False); pass index=False to avoid writing the index.

Q: What does df.head() do?
A: Returns the first 5 rows by default; pass n to control the count.

Q: What does df.tail(n) do?
A: Returns the last n rows (default 5).

Q: How do you get the shape (rows, columns) of a DataFrame?
A: df.shape returns a tuple (n_rows, n_cols).

Q: How do you list the column names?
A: df.columns returns an Index of column labels.

Q: How do you list the row index labels?
A: df.index returns the row index.

Q: What data types does each column have?
A: df.dtypes shows the dtype of every column.

Q: How do you get a concise summary of a DataFrame?
A: df.info() prints shape, dtypes, non-null counts, and memory usage.

Q: How do you select a single column?
A: df['col'] or df.col; the result is a Series.

Q: How do you select multiple columns?
A: Pass a list: df[['col1', 'col2']] returns a DataFrame.

Q: What is loc used for?
A: Label-based selection: df.loc[row_labels, col_labels].

Q: What is iloc used for?
A: Integer position-based selection: df.iloc[row_positions, col_positions].

Q: How do you select the first row with iloc?
A: df.iloc[0] returns the first row as a Series.

Q: How do you select rows 5 through 9 with iloc?
A: df.iloc[5:10] (end is exclusive).

Q: How do you select a single cell by label with loc?
A: df.loc['row_label', 'col_name'].

Q: How do you select a single cell by integer position?
A: df.iat[2, 3] (fast scalar access).

Q: What does df.at do?
A: Access a single scalar value by label, e.g. df.at['row', 'col']; faster than loc for scalars.

Q: How do you filter rows where a column equals a value?
A: df[df['col'] == value] or df.loc[df['col'] == value].

Q: How do you filter rows with multiple AND conditions?
A: Combine boolean masks with &: df[(df['a']>0) & (df['b']<10)].

Q: How do you filter rows with OR conditions?
A: Use |: df[(df['a']>10) | (df['b']<0)]. Each mask must be parenthesized.

Q: How do you filter rows where a column is in a list?
A: Use isin: df[df['col'].isin(['a','b','c'])].

Q: How do you filter rows with NULL values in a column?
A: df[df['col'].isna()]; use notna() for the inverse.

Q: How do you filter rows by string contents?
A: df[df['col'].str.contains('pattern', regex=True)].

Q: How do you query a DataFrame with a string expression?
A: df.query('a > 0 and b < 10'); supports and/or and column references without df..

Q: How do you add a new column?
A: Assign it: df['new'] = df['a'] + df['b'].

Q: How do you delete a column?
A: df.drop('col', axis=1) or del df['col'] (in place).

Q: How do you rename columns?
A: df.rename(columns={'old': 'new'}, inplace=True) or pass a function: df.rename(columns=str.lower).

Q: How do you change a column's dtype?
A: df['col'] = df['col'].astype(int); common targets: int, float, str, 'category', datetime64[ns].

Q: What is the difference between inplace=True and reassignment?
A: df.drop('col', axis=1, inplace=True) modifies df directly; df = df.drop('col', axis=1) rebinds the name to a new object.

Q: How do you sort a DataFrame by column values?
A: df.sort_values('col'); descending: df.sort_values('col', ascending=False).

Q: How do you sort by multiple columns?
A: df.sort_values(['a', 'b'], ascending=[True, False]).

Q: How do you sort by index?
A: df.sort_index(); pass ascending=False to reverse.

Q: How do you reset the index to default integers?
A: df.reset_index(drop=True); drop=True discards the old index.

Q: How do you set a column as the index?
A: df.set_index('col'); use inplace=True to mutate or assign the result.

Q: How do you compute summary statistics per column?
A: df.describe() returns count, mean, std, min, quartiles, max for numeric columns.

Q: How do you compute the mean of each column?
A: df.mean() (numeric columns only; ignores NaN by default).

Q: How do you compute the mean of each row?
A: df.mean(axis=1).

Q: What does axis=0 mean in aggregation?
A: Operate down the rows (compute one value per column); this is the default.

Q: What does axis=1 mean in aggregation?
A: Operate across the columns (compute one value per row).

Q: How do you count non-null values per column?
A: df.count().

Q: How do you count unique values in a column?
A: df['col'].nunique() or df['col'].value_counts() for frequencies.

Q: How do you get unique values of a column?
A: df['col'].unique() returns a NumPy array of distinct values.

Q: What does value_counts() do?
A: Returns a Series with the frequency of each unique value, sorted descending by default.

Q: How do you apply a function to each column?
A: df.apply(np.sum) aggregates per column; pass axis=1 to apply per row.

Q: What is df.applymap() (Pandas < 2.1)?
A: Applies a function element-wise to every cell of a DataFrame. In Pandas 2.1+ it is renamed df.map().

Q: How do you apply a function element-wise to a Series?
A: df['col'].map(func) or df['col'].apply(func); map also accepts a dict/Series for value substitution.

Q: How do you replace values matching a dict?
A: df.replace({'old1': 'new1', 'old2': 'new2'}).

Q: How do you drop rows with any missing value?
A: df.dropna(); pass how='all' to drop only rows that are entirely NaN.

Q: How do you drop columns with missing values?
A: df.dropna(axis=1).

Q: How do you fill NaN with a constant?
A: df.fillna(0); pass a dict to fill per column: df.fillna({'a': 0, 'b': -1}).

Q: How do you forward-fill missing values?
A: df.fillna(method='ffill') (deprecated) or df.ffill() in newer Pandas.

Q: How do you backward-fill missing values?
A: df.bfill() (or fillna(method='bfill')).

Q: How do you check which cells are NaN?
A: df.isna() returns a boolean DataFrame of the same shape.

Q: How do you count NaNs per column?
A: df.isna().sum().

Q: How do you detect duplicate rows?
A: df.duplicated() returns a boolean Series; df.duplicated().sum() counts them.

Q: How do you drop duplicate rows?
A: df.drop_duplicates(); pass subset=['col1','col2'] to consider only specific columns.

Q: How do you group by one column and compute the mean of others?
A: df.groupby('key').mean(numeric_only=True).

Q: How do you group by multiple columns?
A: df.groupby(['k1','k2']).sum() produces a MultiIndex result.

Q: How do you iterate over groups?
A: for name, group in df.groupby('key'): ....

Q: How do you apply multiple aggregations per group?
A: df.groupby('key').agg({'a': 'mean', 'b': ['min','max']}).

Q: How do you name aggregation result columns?
A: Use named aggregation: df.groupby('key').agg(mean_a=('a','mean'), max_b=('b','max')).

Q: What is transform() after groupby?
A: Returns a Series/DataFrame aligned to the original rows, e.g. df.groupby('key')['v'].transform('mean').

Q: What is filter() after groupby?
A: Drops entire groups based on a condition, e.g. df.groupby('key').filter(lambda g: g['v'].mean() > 0).

Q: How do you concatenate DataFrames vertically (stack rows)?
A: pd.concat([df1, df2]) or pd.concat([df1, df2], ignore_index=True) to renumber rows.

Q: How do you concatenate horizontally (add columns)?
A: pd.concat([df1, df2], axis=1).

Q: How do you merge two DataFrames like a SQL join?
A: pd.merge(df1, df2, on='key'); supports how='left'/'right'/'inner'/'outer'.

Q: What is the default join type in pd.merge?
A: inner join — only rows with matching keys in both frames are kept.

Q: How do you merge on different column names?
A: pd.merge(df1, df2, left_on='id1', right_on='id2').

Q: What is the difference between merge and join?
A: join is a convenience method that merges on the index by default (df1.join(df2)); merge is more flexible and works on any columns.

Q: How do you merge on index?
A: df1.join(df2) or pd.merge(df1, df2, left_index=True, right_index=True).

Q: How do you reshape long-to-wide with pivot?
A: df.pivot(index='id', columns='var', values='val'); requires unique index/column pairs.

Q: What does pivot_table do that pivot does not?
A: Aggregates duplicates via an aggregation function (default mean): df.pivot_table(index='id', columns='var', values='val', aggfunc='sum').

Q: How do you reshape wide-to-long with melt?
A: df.melt(id_vars=['id'], var_name='var', value_name='val').

Q: What does stack do?
A: Compresses a level of the column index into the row index, producing a Series with a MultiIndex.

Q: What does unstack do?
A: The inverse of stack: moves a row-index level into the columns.

Q: How do you create a cross-tabulation of two factors?
A: pd.crosstab(df['a'], df['b']); pass values= and aggfunc= to summarize another column.

Q: How do you bin continuous values into discrete buckets?
A: pd.cut(df['v'], bins=[0,10,20,100]) returns a categorical Series; or pd.qcut(df['v'], q=4) for equal-frequency quartiles.

Q: How do you compute a rolling mean over a window of 3?
A: df['v'].rolling(window=3).mean(); the first window-1 values are NaN.

Q: How do you compute an expanding mean (cumulative up to each row)?
A: df['v'].expanding().mean().

Q: How do you compute the difference between consecutive rows?
A: df['v'].diff(); the first row is NaN. Use diff(periods=k) for lag k.

Q: How do you compute the percent change between consecutive rows?
A: df['v'].pct_change(); returns (v_t - v_{t-1}) / v_{t-1}.

Q: How do you shift a Series forward or backward in time?
A: df['v'].shift(1) shifts down (previous value); shift(-1) shifts up.

Q: How do you rank values in a column?
A: df['v'].rank(method='average'); methods include average, min, max, first, dense.

Q: How do you compute the correlation matrix between numeric columns?
A: df.corr(numeric_only=True); pairwise Pearson correlation.

Q: How do you compute pairwise covariance?
A: df.cov(numeric_only=True).

Q: How do you standardize a numeric column (z-score)?
A: (df['v'] - df['v'].mean()) / df['v'].std().

Q: How do you detect outliers using the IQR rule?
A: q1 = df['v'].quantile(0.25); q3 = df['v'].quantile(0.75); iqr = q3 - q1; mask = (df['v'] < q1 - 1.5*iqr) | (df['v'] > q3 + 1.5*iqr).

Q: How do you sample random rows from a DataFrame?
A: df.sample(n=5) for 5 rows; df.sample(frac=0.1, random_state=42) for 10% of rows.

Q: How do you get a reproducible random sample?
A: Pass random_state: df.sample(n=5, random_state=0).

Q: How do you iterate over rows?
A: for index, row in df.iterrows(): ...; returns each row as a Series. Slow for large frames — prefer vectorized operations.

Q: How do you iterate over rows as namedtuples?
A: for row in df.itertuples(): ...; faster than iterrows.

Q: How do you find the index of the maximum value per column?
A: df.idxmax(); df['col'].idxmax() for a single column.

Q: How do you find the position of the maximum value?
A: df['col'].argmax() (deprecated in 2.0); use df['col'].to_numpy().argmax().

Q: How do you clip values to a min/max range?
A: df['v'].clip(lower=0, upper=100).

Q: How do you replace infinities with NaN?
A: df.replace([np.inf, -np.inf], np.nan).

Q: How do you check whether two DataFrames are equal?
A: df1.equals(df2); df1 == df2 returns an element-wise boolean DataFrame.

Q: How do you copy a DataFrame?
A: df.copy() creates a deep copy; assigning without copy can lead to SettingWithCopyWarning.

Q: What is the SettingWithCopyWarning?
A: A warning emitted when chained assignment may not modify the original DataFrame, e.g. df[df['a']>0]['b'] = 1; use df.loc[df['a']>0, 'b'] = 1 instead.

Q: How do you set values using label-based indexing safely?
A: df.loc[mask, 'col'] = new_value; avoids the SettingWithCopyWarning.

Q: What is a categorical dtype and why use it?
A: df['c'] = df['c'].astype('category') stores string columns efficiently and enables ordered comparisons; saves memory and speeds up groupby.

Q: How do you order categories?
A: df['c'] = pd.Categorical(df['c'], categories=['low','med','high'], ordered=True).

Q: How do you convert a column to datetime?
A: pd.to_datetime(df['date_str']); pass format='%Y-%m-%d' for speed.

Q: How do you extract year/month/day from a datetime column?
A: df['date'].dt.year, .dt.month, .dt.day, .dt.dayofweek, .dt.hour, etc.

Q: How do you set a datetime column as the index for time-series work?
A: df.set_index('date', inplace=True); then use df.loc['2024-01'] for partial-string slicing.

Q: How do you resample a time series to a different frequency?
A: df.resample('M').mean() aggregates monthly; frequencies: 'D', 'W', 'M', 'Q', 'Y', 'H'.

Q: How do you compute a 7-day rolling sum on a DatetimeIndex?
A: df.rolling('7D').sum(); '7D' is a time-based window.

Q: How do you concatenate two columns into one string column?
A: df['full'] = df['first'] + ' ' + df['last']; or df['full'] = df[['first','last']].agg(' '.join, axis=1).

Q: How do you split a string column by a delimiter?
A: df['col'].str.split(',', expand=True) returns a DataFrame of parts.

Q: How do you extract a regex capture group from a string column?
A: df['col'].str.extract(r'(\d+)') returns a DataFrame with one column per group.

Q: How do you strip whitespace from a string column?
A: df['col'].str.strip(); variants: lstrip(), rstrip().

Q: How do you replace substrings in a string column?
A: df['col'].str.replace('old', 'new', regex=False); pass regex=True for patterns.

Q: How do you get the length of each string?
A: df['col'].str.len().

Q: How do you check whether string values start with a prefix?
A: df['col'].str.startswith('pre'); also endswith and contains.