ASAmol Shukla
Projects
Learning Hub
Prompts
Skills
Contact
Resume
Course Outline
Syllabus Overview

Data Science: NumPy, pandas, Matplotlib, seaborn & Plotly

Courses/Data Science: NumPy, pandas, Matplotlib, seaborn & Plotly/2: NumPy Arrays & Pandas Wrangling

Data Science

2: NumPy Arrays & Pandas Wrangling

N-dimensional arrays, vectorized functions, DataFrame operations, grouping, and handling missing data.

60 mins8 mins read

NumPy and the Art of Vectorization

Underneath standard Python lists lies cell pointers, making looping slow. NumPy introduces the ndarray—a contiguous block of homogeneous memory that delegates operations to highly optimized C/Fortran libraries.

What is Vectorization?

Instead of writing explicit loops to apply an operation to every element of an array, vectorized functions perform operations on the entire array at once:

  • Loop approach: $O(N)$ overhead of Python runtime type checks.
  • Vectorized approach: Performed in a single CPU instruction pass.

Pandas DataFrames

Pandas extends NumPy by adding labels and indexes, creating the DataFrame. It is the ultimate tool for handling tabular data.

Essential Wrangling Techniques

  1. Handling Missing Values:
    • Impute/fill missing entries with the column mean: df.fillna(df.mean())
    • Drop rows with excessive missing data: df.dropna(subset=['critical_column'])
  2. Indexing & Slicing:
    • Use $.loc$ for label-based indexing.
    • Use $.iloc$ for integer-position indexing.
  3. Aggregations & GroupBy:
    • Group data by categorical fields and calculate statistical aggregates (mean, sum, median).
python · worked example
import numpy as np
import pandas as pd

# 1. NumPy Vectorized operations
arr = np.array([1, 2, 3, 4, 5])
print("Vectorized multiplication (arr * 10):", arr * 10)

# 2. Pandas Data Wrangling
data = {
    'Department': ['IT', 'HR', 'IT', 'Marketing', 'HR', 'IT'],
    'Salary': [85000, 60000, 95000, 70000, None, 90000],
    'Experience': [3, 2, 5, 4, 1, 4]
}
df = pd.DataFrame(data)

# Impute missing salary values with the mean of the column
mean_salary = df['Salary'].mean()
df['Salary'] = df['Salary'].fillna(mean_salary)

# Perform Groupby aggregation
dept_summary = df.groupby('Department').agg(
    Average_Salary=('Salary', 'mean'),
    Total_Employees=('Salary', 'count'),
    Avg_Experience=('Experience', 'mean')
).round(2)

print("
Original DataFrame with Imputed Salary:")
print(df)
print("
Department Summary Analysis:")
print(dept_summary)
Read-only sample

Lesson Code (python)

import numpy as np
import pandas as pd

# 1. NumPy Vectorized operations
arr = np.array([1, 2, 3, 4, 5])
print("Vectorized multiplication (arr * 10):", arr * 10)

# 2. Pandas Data Wrangling
data = {
    'Department': ['IT', 'HR', 'IT', 'Marketing', 'HR', 'IT'],
    'Salary': [85000, 60000, 95000, 70000, None, 90000],
    'Experience': [3, 2, 5, 4, 1, 4]
}
df = pd.DataFrame(data)

# Impute missing salary values with the mean of the column
mean_salary = df['Salary'].mean()
df['Salary'] = df['Salary'].fillna(mean_salary)

# Perform Groupby aggregation
dept_summary = df.groupby('Department').agg(
    Average_Salary=('Salary', 'mean'),
    Total_Employees=('Salary', 'count'),
    Avg_Experience=('Experience', 'mean')
).round(2)

print("
Original DataFrame with Imputed Salary:")
print(df)
print("
Department Summary Analysis:")
print(dept_summary)

Console Output

Vectorized multiplication (arr * 10): [10 20 30 40 50]

Original DataFrame with Imputed Salary:
  Department   Salary  Experience
0         IT  85000.0           3
1         HR  60000.0           2
2         IT  95000.0           5
3  Marketing  70000.0           4
4         HR  80000.0           1
5         IT  90000.0           4

Department Summary Analysis:
            Average_Salary  Total_Employees  Avg_Experience
Department                                                 
HR                 70000.0                2             1.5
IT                 90000.0                3             4.0
Marketing          70000.0                1             4.0

Up next · Continue learning

Visualizing Patterns with Seaborn & Matplotlib

Building distribution charts, relational scatter plots, and correlation heatmaps.

6 mins read50 mins
Start next lesson
Previous: The Python Data Science EcosystemNext: Visualizing Patterns with Seaborn & Matplotlib
Made withbyAmol Shukla·amolshukla.online
ASAmol Shukla

AI Developer, Trainer & Agentic AI Expert building practical learning systems and real-world AI applications.

Explore

  • Projects
  • Learning Hub
  • Prompts
  • Skills
  • Contact
  • Experience
  • Blogs

Connect

  • Resume
  • Contact
© 2026 Amol Shukla·Created withbyamolshukla.online
Back to top