Unit-3: Evaluating Models

 

Q1. What will happen if you deploy an AI model without evaluating it with known test set data?
Answer: Deploying an AI model without evaluation risks severe problems: the model may be overfitted to the training data and perform poorly on real, unseen data. Evaluation with a known test set estimates how well the model generalizes, identifies weaknesses (e.g., class imbalance, high false negative rate), and provides guidance for improvement. In short: evaluation is essential before deployment; it is the model’s ‘report card.’

Q2. Is evaluating an AI model essential in the AI project cycle?
Answer: Yes. Model evaluation is an integral stage in the AI project cycle. It gives objective feedback about model performance, informs decisions about choosing or tuning models, and helps ensure that the model will behave reliably in real-world settings.

Q3. Explain train-test split with an example.
Answer: Train-test split divides the dataset into two parts: a training set (used to train the model) and a test set (used to evaluate it). Example: Given 1,000 labeled samples, use 800 (80%) for training and 200 (20%) for testing. Train the model on the 800 samples; then make predictions for the 200 test samples and compare predictions to true labels to estimate performance on new data. This technique helps detect overfitting.

Q4. “Understanding both error and accuracy is crucial”—justify.
Answer: Accuracy measures the fraction of correct predictions, while error quantifies mistakes or deviations. A model might have high accuracy on imbalanced data but still miss critical minority-class cases (high error for that class). Understanding both helps choose appropriate metrics (precision/recall/F1) and guides targeted improvements.

Q5. What is classification accuracy? Can it be used all the time?
Answer: Classification accuracy = (TP + TN) / Total. It is useful but not always appropriate—particularly for imbalanced datasets. In such cases, precision, recall or F1-score better reflect model usefulness.

Assertion–Reasoning

Q1. A: ‘Accuracy measures total correct predictions.’ R: ‘Accuracy is directly proportional to model performance.’
Answer: (a) Both A and R are true and R is the correct explanation for A.
Explanation: Accuracy counts correct predictions; rising accuracy generally indicates better performance at overall correctness. However, for imbalanced classes, accuracy alone can be misleading.

Q2. A: ‘Sum of the values in a confusion matrix’s row represents total instances for an actual class.’ R: ‘This enables calculation of class-specific metrics such as precision and recall.’
Answer: (b) Both A and R are true, but R is not the correct explanation for A.
Explanation: The row sum indeed equals total actual instances of that class (this directly supports calculations of recall since recall uses TP / (TP + FN) where denominator is row total). Precision uses predicted-class column totals, so the reasoning is partially related but not the direct explanation.