🔢 Introduction to NumPy

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

Why NumPy?

NumPy arrays are 10–100× faster than Python lists for numerical operations. It's the foundation of pandas, matplotlib, and scikit-learn.

python
import numpy as np # Python list — slow for math py_list = [1, 2, 3, 4, 5] # NumPy array — fast vectorized operations arr = np.array([1, 2, 3, 4, 5]) # NumPy does math on ALL elements at once print(arr * 2) # [ 2 4 6 8 10] print(arr + 10) # [11 12 13 14 15] print(arr ** 2) # [ 1 4 9 16 25] print(arr * arr) # [ 1 4 9 16 25] (element-wise) # Python list — can't do this directly # py_list * 2 → [1,2,3,4,5,1,2,3,4,5] (repeats list!)
Output
[ 2 4 6 8 10] [11 12 13 14 15] [ 1 4 9 16 25] [ 1 4 9 16 25]

Creating Arrays

python
import numpy as np zeros = np.zeros(5) # [0. 0. 0. 0. 0.] ones = np.ones((2,3)) # 2x3 matrix of ones rng = np.arange(0, 10, 2) # [0 2 4 6 8] linear = np.linspace(0, 1, 5) # [0. .25 .5 .75 1.] eye = np.eye(3) # 3x3 identity matrix full = np.full((2,2), 7) # 2x2 matrix of 7s print(rng) print(linear) print(full)
Output
[0 2 4 6 8] [0. 0.25 0.5 0.75 1. ] [[7 7] [7 7]]
🎯 TASK Practice: Introduction to NumPy
Create 5 different arrays using NumPy functions. Calculate: element-wise profit (sales - costs), percentage margin, and identify months where margin > 30%.
Starter code:
python
import numpy as np sales = np.array([45000, 52000, 48000, 61000, 58000, 72000]) costs = np.array([32000, 36000, 35000, 40000, 38000, 45000]) profit = ??? margin = ??? # as percentage above30 = ??? # boolean mask print("Profit:", profit) print("Margin:", margin.round(1)) print("Months > 30%:", np.where(above30)[0] + 1) print("Avg margin:", margin.mean().round(1), "%")

🔢 Arrays & Dimensions

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

1D, 2D, and 3D Arrays

python
import numpy as np # 1D — like a list sales_week = np.array([45000, 52000, 48000, 61000, 58000]) print("1D shape:", sales_week.shape) # (5,) # 2D — rows × columns (like a table) sales_matrix = np.array([ [45000, 52000, 48000], # Week 1: 3 regions [61000, 58000, 72000], # Week 2: 3 regions [55000, 49000, 68000], # Week 3: 3 regions ]) print("2D shape:", sales_matrix.shape) # (3, 3) print("Element [1,2]:", sales_matrix[1, 2]) # 72000 print("Row 0:", sales_matrix[0]) print("Col 1:", sales_matrix[:, 1]) # All rows, col 1
Output
1D shape: (5,) 2D shape: (3, 3) Element [1,2]: 72000 Row 0: [45000 52000 48000] Col 1: [52000 58000 49000]

Array Properties & Slicing

python
data = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12]]) print("shape:", data.shape) # (3, 4) print("ndim: ", data.ndim) # 2 print("size: ", data.size) # 12 print("dtype:", data.dtype) # int64 # Advanced slicing print(data[0:2, 1:3]) # rows 0-1, cols 1-2 print(data[::2, ::2]) # every 2nd row/col
Output
shape: (3, 4) ndim: 2 size: 12 dtype: int64 [[2 3] [6 7]] [[ 1 3] [ 9 11]]
🎯 TASK Practice: Arrays & Dimensions
Create a 3×4 sales matrix (3 regions × 4 quarters). Calculate total per region (row sum), total per quarter (col sum), and the region/quarter with maximum sales.
Starter code:
python
import numpy as np sales = np.array([ [120000, 135000, 125000, 160000], # Cairo [80000, 92000, 88000, 105000], # Alex [55000, 61000, 58000, 72000], # Giza ]) region_names = ["Cairo", "Alex", "Giza"] quarter_names = ["Q1","Q2","Q3","Q4"] region_totals = np.sum(sales, axis=???) # sum each row quarter_totals = np.sum(sales, axis=???) # sum each column best_region = region_names[np.argmax(region_totals)] best_quarter = quarter_names[np.argmax(quarter_totals)] for name, total in zip(region_names, region_totals): print(f"{name}: {total:,}") print(f"Best region: {best_region}") print(f"Best quarter: {best_quarter}")

🔢 Operations & Filtering

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

Element-wise Operations

python
import numpy as np sales = np.array([45000, 52000, 48000, 61000, 58000]) costs = np.array([30000, 35000, 32000, 40000, 38000]) # All operations are element-wise profit = sales - costs margin = (profit / sales * 100).round(1) growth = np.diff(sales) # difference between consecutive print("Profit:", profit) print("Margins:", margin) print("Month-to-month change:", growth)
Output
Profit: [15000 17000 16000 21000 20000] Margins: [33.3 32.7 33.3 34.4 34.5] Month-to-month change: [ 7000 -4000 13000 -3000]

Boolean Filtering

python
sales = np.array([45000, 52000, 28000, 61000, 19000, 58000]) # Create boolean mask mask = sales > 40000 print("Mask:", mask) # Apply mask — select elements high = sales[mask] print("Above 40k:", high) # Combined conditions medium = sales[(sales > 30000) & (sales < 60000)] print("Medium:", medium) # np.where — conditional assignment labels = np.where(sales > 50000, "High", "Low") print("Labels:", labels) # Count below_target = np.sum(sales < 50000) print(f"{below_target} of {len(sales)} below target")
Output
Mask: [ True True False True False True] Above 40k: [45000 52000 61000 58000] Medium: [45000 52000 58000] Labels: ['Low' 'High' 'Low' 'High' 'Low' 'High'] 3 of 6 below target
🎯 TASK Practice: Operations & Filtering
Analyze product prices: apply tiered discount (>10000→25%, >5000→15%, else→5%), flag premium items, calculate savings per item.
Starter code:
python
import numpy as np prices = np.array([12000, 6500, 3400, 9800, 800, 15000, 4200]) # Tiered discount using np.where discounts = np.where(prices > 10000, 0.25, np.where(prices > 5000, 0.15, 0.05)) final_prices = prices * (1 - discounts) savings = prices - final_prices is_premium = prices > 10000 for i in range(len(prices)): flag = "⭐" if is_premium[i] else " " print(f"{flag} {prices[i]:>6,} → {final_prices[i]:>8,.0f} (save {savings[i]:,.0f})") print(f"\nTotal saved: {savings.sum():,.0f}")

🔢 Statistics with NumPy

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

Descriptive Statistics

python
import numpy as np data = np.array([45000,52000,48000,61000,58000,72000,31000,55000]) print(f"Count: {len(data)}") print(f"Sum: {np.sum(data):,}") print(f"Mean: {np.mean(data):,.0f}") print(f"Median: {np.median(data):,.0f}") print(f"Std: {np.std(data):,.0f}") print(f"Var: {np.var(data):,.0f}") print(f"Min: {np.min(data):,}") print(f"Max: {np.max(data):,}") print(f"Range: {np.ptp(data):,}") # peak-to-peak
Output
Count: 8 Sum: 422,000 Mean: 52,750 Median: 53,500 Std: 12,113 Var: 146,734,375 Min: 31,000 Max: 72,000 Range: 41,000

Percentiles & Correlation

python
import numpy as np sales = np.array([45000,52000,48000,61000,58000,72000,31000,55000]) # Percentiles p25 = np.percentile(sales, 25) # Q1 p50 = np.percentile(sales, 50) # Median p75 = np.percentile(sales, 75) # Q3 iqr = p75 - p25 print(f"Q1: {p25:,} Q3: {p75:,} IQR: {iqr:,}") # Correlation between two variables marketing = np.array([10,25,15,40,20,35,50,30]) corr = np.corrcoef(marketing, sales)[0,1] print(f"Correlation: {corr:.3f}")
Output
Q1: 47,250 Q3: 59,250 IQR: 12,000 Correlation: 0.923
🎯 TASK Practice: Statistics with NumPy
Given exam scores for 30 students (use np.random.seed for reproducibility), calculate full stats, identify students above/below 1 std dev from mean, and build a grade distribution.
Starter code:
python
import numpy as np np.random.seed(42) scores = np.random.randint(40, 100, 30) mean = np.mean(scores) std = np.std(scores) above_1std = scores[scores > mean + std] below_1std = scores[scores < mean - std] # Grade distribution grades = { "A (90+)": np.sum(scores >= 90), "B (80-89)": np.sum((scores>=80)&(scores<90)), "C (70-79)": np.sum((scores>=70)&(scores<80)), "D (60-69)": np.sum((scores>=60)&(scores<70)), "F (<60)": np.sum(scores < 60), } print(f"Mean: {mean:.1f} | Std: {std:.1f}") print(f"Above +1std: {len(above_1std)} students") print(f"Below -1std: {len(below_1std)} students") for grade, count in grades.items(): bar = "█" * count print(f"{grade}: {bar} ({count})")

🔢 Reshape, Sort & Search

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

Reshape

python
import numpy as np data = np.arange(12) # [0,1,...,11] table = data.reshape(3, 4) # 3 rows × 4 cols flat = table.flatten() # back to 1D (copy) view = table.ravel() # back to 1D (view, faster) print("Original:", data) print("Reshaped:\n", table) # -1 = auto-calculate dimension auto = data.reshape(-1, 4) # auto rows, 4 cols print("Auto rows:", auto.shape) # (3, 4) # Transpose print("Transposed shape:", table.T.shape) # (4, 3)
Output
Original: [ 0 1 2 3 4 5 6 7 8 9 10 11] Reshaped: [[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11]] Auto rows: (3, 4) Transposed shape: (4, 3)

Sort & Search

python
sales = np.array([45000, 72000, 31000, 58000, 19000, 61000]) print("Sorted: ", np.sort(sales)) print("Sorted desc: ", np.sort(sales)[::-1]) print("Min index: ", np.argmin(sales)) # 4 print("Max index: ", np.argmax(sales)) # 1 # argsort — indices that would sort order = np.argsort(sales) print("Rank order:", order) # np.where — find indices idx = np.where(sales > 50000)[0] print("Above 50k indices:", idx) print("Above 50k values:", sales[idx])
Output
Sorted: [19000 31000 45000 58000 61000 72000] Sorted desc: [72000 61000 58000 45000 31000 19000] Min index: 4 Max index: 1 Rank order: [4 2 0 3 5 1] Above 50k indices: [1 3 5] Above 50k values: [72000 58000 61000]
🎯 TASK Practice: Reshape, Sort & Search
Create a 12-month sales array. Reshape to quarters (3 months each), find best quarter, sort months by sales and print ranking.
Starter code:
python
import numpy as np monthly = np.array([45,52,48,61,58,72,55,49,67,71,63,80]) * 1000 # Reshape to 4 quarters x 3 months quarters = monthly.reshape(4, 3) q_totals = np.sum(quarters, axis=1) best_q = np.argmax(q_totals) + 1 print(f"Quarterly totals: {q_totals}") print(f"Best quarter: Q{best_q} ({q_totals[best_q-1]:,})") # Rank months (1=highest) months = ["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"] ranking = np.argsort(monthly)[::-1] print("\nMonthly ranking:") for rank, idx in enumerate(ranking, 1): print(f" {rank:2}. {months[idx]}: {monthly[idx]:,}")

🔢 Random Numbers

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

NumPy Random Module

python
import numpy as np # Set seed for reproducibility np.random.seed(42) # Uniform [0, 1) print(np.random.random(5)) # Random integers print(np.random.randint(1, 101, size=5)) # 1-100 # Normal distribution (mean=0, std=1) print(np.random.normal(0, 1, 5).round(2)) # Normal with custom mean & std salaries = np.random.normal(loc=9000, scale=2000, size=100) print(f"Mean: {salaries.mean():.0f}, Std: {salaries.std():.0f}") # Random choice from array items = ["Cairo", "Alex", "Giza", "Aswan"] print(np.random.choice(items, size=3, replace=False))
Output
[0.37 0.95 0.73 0.60 0.16] [52 93 15 72 61] [-0.47 0.16 0.31 -1.03 0.13] Mean: 8964, Std: 2012 ['Alex' 'Cairo' 'Aswan']

Random in Data Science

python
import numpy as np np.random.seed(42) # Simulate a month of sales daily_sales = np.random.normal(50000, 8000, 30).clip(min=0) print(f"Month total: {daily_sales.sum():,.0f}") print(f"Best day: {daily_sales.max():,.0f}") print(f"Worst day: {daily_sales.min():,.0f}") # Shuffle data (for train/test split) data = np.arange(1, 11) np.random.shuffle(data) print("Shuffled:", data) # Random sample (for testing) sample = np.random.choice(daily_sales, size=7, replace=False) print(f"7-day sample mean: {sample.mean():,.0f}")
💡 Tip: Always set np.random.seed(n) before generating random numbers in notebooks — ensures reproducible results.
🎯 TASK Practice: Random Numbers
Simulate a year of daily transactions. Generate 365 transaction amounts (normal distribution, mean=2000, std=500). Find busiest day, quietest day, and days above 3000.
Starter code:
python
import numpy as np np.random.seed(123) transactions = np.random.normal(loc=2000, scale=500, size=365).clip(min=100) transactions = transactions.round(0) # Stats print(f"Annual total: {transactions.sum():,.0f}") print(f"Daily average: {transactions.mean():,.0f}") print(f"Best day: {transactions.max():,.0f} (day {np.argmax(transactions)+1})") print(f"Worst day: {transactions.min():,.0f} (day {np.argmin(transactions)+1})") # Days above 3000 busy_days = np.sum(transactions > 3000) print(f"Days > 3000: {busy_days} ({busy_days/365*100:.1f}%)") # Monthly totals (reshape 365 days — use first 360) monthly = transactions[:360].reshape(12, 30) monthly_totals = monthly.sum(axis=1) best_month = np.argmax(monthly_totals) + 1 print(f"Best month: Month {best_month} ({monthly_totals[best_month-1]:,.0f})")

🔢 Broadcasting

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

What is Broadcasting?

Broadcasting lets NumPy perform operations between arrays of different shapes — automatically expanding the smaller array to match the larger one.

python
import numpy as np # Scalar + array (simplest broadcasting) arr = np.array([1, 2, 3, 4, 5]) print(arr + 10) # 10 is broadcast to [10,10,10,10,10] # 1D + 2D broadcasting sales = np.array([ [45000, 52000, 48000], # Region 1: 3 months [61000, 58000, 72000], # Region 2: 3 months ]) tax_rates = np.array([0.14, 0.12, 0.10]) # Different rate per month # Broadcasting: (2,3) × (3,) — rate applied to each column taxes = sales * tax_rates print("Taxes:\n", taxes.round(0))
Output
[ 11 12 13 14 15] Taxes: [[ 6300. 6240. 4800.] [ 8540. 6960. 7200.]]

Practical Broadcasting

python
import numpy as np # Normalize each column (subtract mean, divide by std) data = np.array([[10,200,0.5],[20,300,1.2],[30,100,0.8]]) col_mean = data.mean(axis=0) # Shape (3,) col_std = data.std(axis=0) # Shape (3,) # Broadcasting: (3,3) - (3,) → each row gets the column means subtracted normalized = (data - col_mean) / col_std print("Normalized:\n", normalized.round(2)) # Add target to each region targets = np.array([[50000],[60000]]) # Shape (2,1) sales = np.array([45000,52000,48000,61000]) # Shape (4,) # (2,1) vs (4,) → broadcast to (2,4) diff = sales - targets print("\nDiff from targets:\n", diff)
💡 Tip: Broadcasting rule: dimensions are compatible if they're equal OR one of them is 1.
🎯 TASK Practice: Broadcasting
Use broadcasting to: normalize monthly sales per region to 0-1 scale, calculate each month's performance vs its column average, and rank regions by total.
Starter code:
python
import numpy as np sales = np.array([ [45000,52000,48000,61000], [38000,42000,45000,55000], [28000,31000,29000,38000], ]) regions = ["Cairo","Alex","Giza"] # Normalize each row (region) to 0-1 row_min = sales.min(axis=1, keepdims=True) row_max = sales.max(axis=1, keepdims=True) normalized = (sales - row_min) / (row_max - row_min) # Performance vs column mean col_mean = sales.mean(axis=0, keepdims=True) vs_avg = (sales - col_mean) / col_mean * 100 # Rank regions totals = sales.sum(axis=1) ranking = np.argsort(totals)[::-1] print("Normalized (0-1):\n", normalized.round(2)) print("\nVs monthly avg (%):\n", vs_avg.round(1)) print("\nRegion ranking:", [regions[i] for i in ranking])

🔢 Linear Algebra Basics

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

Matrix Operations

python
import numpy as np A = np.array([[1,2],[3,4]]) B = np.array([[5,6],[7,8]]) # Matrix multiplication (dot product) print("A @ B:") print(A @ B) # Matrix multiply # Element-wise (NOT matrix multiply) print("A * B:") print(A * B) # Element-wise # Transpose print("A.T:") print(A.T) # Determinant and inverse print("det(A):", np.linalg.det(A).round(2)) A_inv = np.linalg.inv(A) print("inv(A):") print(A_inv.round(2))
Output
A @ B: [[19 22] [43 50]] A * B: [[ 5 12] [21 32]] A.T: [[1 3] [2 4]] det(A): -2.0 inv(A): [[-2. 1. ] [ 1.5 -0.5]]

Practical: Solve Linear System

python
import numpy as np # Solve: 2x + y = 5, x + 3y = 10 # As matrix: A @ [x,y] = b A = np.array([[2, 1],[1, 3]]) b = np.array([5, 10]) solution = np.linalg.solve(A, b) print(f"x = {solution[0]:.2f}, y = {solution[1]:.2f}") # Verify: A @ solution ≈ b print("Verify:", np.allclose(A @ solution, b)) # Eigenvalues (used in PCA) eigenvalues, eigenvectors = np.linalg.eig(A) print("Eigenvalues:", eigenvalues.round(2))
Output
x = 1.00, y = 3.00 Verify: True Eigenvalues: [2.27 2.73]
🎯 TASK Practice: Linear Algebra Basics
Use linear algebra to solve a portfolio optimization problem: given returns matrix and target return, find weights using least squares.
Starter code:
python
import numpy as np np.random.seed(42) # Simulate 4 assets × 12 months of returns returns = np.random.normal(0.05, 0.02, (12, 4)) # Covariance matrix (used in portfolio theory) cov_matrix = np.cov(returns.T) print("Covariance matrix shape:", cov_matrix.shape) print("\nAsset correlations:") corr = np.corrcoef(returns.T) print(corr.round(2)) # Simple equal-weight portfolio weights = np.array([0.25, 0.25, 0.25, 0.25]) portfolio_return = returns @ weights print(f"\nPortfolio monthly returns (first 6):") print(portfolio_return[:6].round(4)) print(f"Average return: {portfolio_return.mean():.4f}") print(f"Risk (std): {portfolio_return.std():.4f}")

🔢 Date & Time with NumPy

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

NumPy Datetime64

NumPy uses datetime64 for fast date/time operations on large arrays.

python
import numpy as np # Monthly dates months = np.array(["2024-01","2024-02","2024-03"], dtype="datetime64[M]") # Daily date range days = np.arange("2024-01-01","2024-01-08", dtype="datetime64[D]") # Add 1 month to each next_m = months + np.timedelta64(1,"M") print(months) print(days)

Business Days

python
import numpy as np all_days = np.arange("2024-01-01","2025-01-01", dtype="datetime64[D]") biz_days = all_days[np.is_busday(all_days)] print(f"Total days: {len(all_days)}") print(f"Business days: {len(biz_days)}") today = np.datetime64("2024-06-15") end = np.datetime64("2024-12-31") days_left = int((end - today) / np.timedelta64(1,"D")) print(f"Days remaining: {days_left}")
Output
Total days: 366 Business days: 262 Days remaining: 199
🎯 TASK Practice: Date & Time with NumPy
Generate Q4 2024 dates, filter to business days, count per month.
Starter code:
python
import numpy as np q4 = np.arange("2024-10-01","2025-01-01",dtype="datetime64[D]") biz = q4[np.is_busday(q4)] print(f"Q4 total: {len(q4)}, business: {len(biz)}") for m in ["2024-10","2024-11","2024-12"]: ms=np.datetime64(m); me=ms+np.timedelta64(1,"M") days=q4[(q4>=ms)&(q4<me)] print(f"{m}: {np.sum(np.is_busday(days))} biz days")

🔢 Performance Tips

Unit 4 — Fast numerical computing with N-dimensional arrays — the engine behind all data science in Python.

NumPy vs Python Speed

python
import numpy as np import time n = 1_000_000 data = list(range(n)) arr = np.array(data) # Python loop — slow start = time.time() result_py = [x ** 2 for x in data] py_time = time.time() - start # NumPy vectorized — fast start = time.time() result_np = arr ** 2 np_time = time.time() - start print(f"Python: {py_time:.3f}s") print(f"NumPy: {np_time:.3f}s") print(f"NumPy is {py_time/np_time:.0f}x faster!")
Output
Python: 0.185s NumPy: 0.003s NumPy is 62x faster!

Memory Efficiency

python
import numpy as np import sys py_list = list(range(10000)) np_arr = np.array(py_list) np_f32 = np.array(py_list, dtype=np.float32) # 32-bit float print(f"Python list: {sys.getsizeof(py_list):,} bytes") print(f"NumPy int64: {np_arr.nbytes:,} bytes") print(f"NumPy float32: {np_f32.nbytes:,} bytes") # Available dtypes # int8, int16, int32, int64 # float32, float64 # bool, complex64, complex128
Output
Python list: 87,624 bytes NumPy int64: 80,000 bytes NumPy float32: 40,000 bytes
💡 Tip: Use float32 instead of float64 when precision isn't critical — uses half the memory and can be 2× faster on some hardware.
🎯 TASK Practice: Performance Tips
Compare performance: compute the dot product of two large vectors (1M elements) using a Python loop, list comprehension, and NumPy. Time each method.
Starter code:
python
import numpy as np import time np.random.seed(42) n = 1_000_000 a = np.random.random(n) b = np.random.random(n) # Method 1: Python loop start = time.time() result1 = sum(a[i]*b[i] for i in range(n)) t1 = time.time() - start # Method 2: NumPy dot start = time.time() result2 = np.dot(a, b) t2 = time.time() - start # Method 3: NumPy @ operator start = time.time() result3 = a @ b t3 = time.time() - start print(f"Python loop: {t1:.3f}s → {result1:.4f}") print(f"NumPy dot: {t2:.4f}s → {result2:.4f}") print(f"NumPy @: {t3:.4f}s → {result3:.4f}") print(f"Speedup: {t1/t2:.0f}x")