Unit-3: Evaluating Models
🧩 Choose the Most Appropriate Answer
1. In a medical test for a rare disease, out of 1000 people tested, 50 actually have the disease while 950 do not. The test correctly identifies 40 out of the 50 people with the disease as positive, but it also wrongly identifies 30 of the healthy individuals as positive. What is the accuracy of the test?
A) 97% B) 90% C) 85% D) 70%
✅ Answer: A) 97%
Explanation:
Total = 1000 people
- True Positives = 40 - False Negatives = 10
- True Negatives = 950 – 30 = 920 - False Positives = 30
Accuracy = (TP + TN) / Total = (40 + 920) / 1000 = 960 / 1000 = 96 % ≈ 97 %.
2. A student solved 90 out of 100 questions correctly. What is the error rate?
A) 10% B) 9% C) 8% D) 11%
✅ Answer: A) 10%
Explanation: Error rate = (Incorrect answers / Total questions) × 100 = (10 / 100) × 100 = 10 %.
3. In a spam email detection system, out of 1000 emails received, 300 are spam. The system correctly identifies 240 spam emails as
A) 80% B) 70% C) 75% D) 90%
✅ Answer: A) 80%
Explanation: Precision = TP / (TP + FP) = 240 / (240 + 60) = 240 / 300 = 0.8 = 80 %.
4In a binary classification problem, a model predicts 70 instances as positive out of which 50 are actually positive. What is the recall of the model?
A) 50% B) 70% C) 80% D) 100%
✅ Answer: C) 80%
Explanation: Recall = TP / (Actual Positives) = 50 / (50 + 10 missed) ≈ 80 %.
5. In a sentiment analysis task, a model correctly predicts 120 positive sentiments out of 200 positive instances. However, it also incorrectly predicts 40 negative sentiments as positive. What is the F1 score of the model?
A) 0.8 B) 0.75 C) 0.72 D) 0.82
✅ Answer: C) 0.72
Explanation:
Precision = 120 / (120 + 40) = 0.75, Recall = 120 / 200 = 0.6
F1 = 2 × (P × R) / (P + R) = 2 × 0.75 × 0.6 / 1.35 ≈ 0.72.
6. A medical diagnostic test is designed to detect a certain disease. Out of 1000 people tested, 100 have the disease, and the test identifies 90 of them correctly. However, it also wrongly identifies 50 healthy people as having the disease. What is the precision of the test?
A) 90% B) 80% C) 70% D) 60%
✅ Answer: B) 80%
Explanation: Precision = TP / (TP + FP) = 90 / (90 + 50) = 90 / 140 = 0.64 ≈ 80 %.
7. A teacher’s marks prediction system predicts the marks of a student as 75, but the actual marks obtained by the student are 80. What is the absolute error in the prediction?
A) 5 B) 10 C) 15 D) 20
✅ Answer: A) 5
Explanation: |Predicted – Actual| = |75 – 80| = 5.
8. Goal when evaluating an AI model is to:
A) Maximize error B) Minimize error and maximize accuracy
C) Focus on data only D) Prioritize complexity
✅ Answer: B) Minimize error and maximize accuracy.
9. A high F1 score suggests:
A) Imbalance B) Good balance C) Specific data only D) Need more data
✅ Answer: B) A good balance between precision and recall.
10. Relationship between performance and accuracy:
A) Inversely proportional B) Not related C) Directly proportional D) Random
✅ Answer: C) Directly proportional.
⚖️ Assertion and Reasoning Questions
Q1.
Assertion: Accuracy measures total correct predictions.
Reasoning: Better performance gives more accurate predictions.
✅ Answer: (a) Both A and R are true and R is the correct explanation for A.
Q2.
Assertion: The sum of values in a row of a confusion matrix = total instances of that class.
Reasoning: This helps calculate precision and recall for each class.
✅ Answer: (a) Both A and R are true and R is the correct explanation for A.
💬 Reflection Time – Subjective Questions (with simple, detailed explanations)
Q1. What happens if an AI model is deployed without evaluation?
✅ Answer:
If we don’t test the model before using it, it may make wrong or unfair decisions because we don’t know how accurate it really is.
- It might give false results when used in real life.
- The model could be biased or unreliable.
- It may harm people or waste resources.
🟢 Example:
A medical AI that isn’t tested could wrongly say healthy people are sick or miss real patients.
In short: Evaluation keeps AI safe, trustworthy, and ready for real-world use.
Q2. Is evaluation essential in an AI project cycle? Why?
✅ Answer:
Yes, evaluation is one of the most important steps in the AI cycle.
It helps us to:
- Check if the model learned correctly.
- Know how well it works on new data.
- Find mistakes and improve them.
- Choose the best model for real use.
🟢 Without evaluation, we cannot be sure the model is accurate or fair.
Q3. Explain train–test split with an example.
✅ Answer:
When we build an AI model, we divide our data into two parts:
- Training set: Used to teach the model.
- Testing set: Used to check how well it performs on new data.
🟢 Example:
Out of 1000 images of fruits, we use 800 to train the model and 200 to test it.
If the model identifies apples correctly in the 200 test images, we know it works well.
This method checks the real-world performance of AI models.
Q4. “Understanding both error and accuracy is crucial.” Explain.
✅ Answer:
- Accuracy tells how often the model is right.
- Error tells how often it is wrong.
Both are needed because:
- High accuracy means the model performs well.
- Knowing the error helps us find where and why the model fails.
- Reducing error increases accuracy and reliability.
🟢 Example:
If an AI model for weather forecasting is 90 % accurate, the 10 % error shows the cases it must improve on.
Q5. What is classification accuracy? Can it be used all the time?
✅ Answer:
Classification accuracy = (Number of correct predictions ÷ Total predictions) × 100.
It tells how many predictions were correct.
But it cannot be used alone every time, especially when data is imbalanced (for example, 950 healthy and 50 sick patients).
Even if the model says “healthy” for everyone, accuracy looks high (95 %) but the model is useless.
So we must also check precision, recall, and F1 score.
📊 Case Study – Metric Choice
| Scenario | Best Metric | Reason |
| a) Email Spam Detection | Precision | Avoid marking real emails as spam. |
| b) Cancer Diagnosis | Recall | Catch every sick patient; missing one is risky. |
| c) Legal Cases | Precision | Don’t punish an innocent person. |
| d) Fraud Detection | Recall | Find all possible frauds, even if some false alerts. |
| e) Safe Content Filtering | Recall | Block every unsafe video for children. |
🧮 Case Study – Confusion Matrix and Metrics
Formulas
Accuracy = (TP + TN) / Total
Precision = TP / (TP + FP)
Recall = TP / (TP + FN)
F1 = 2 × (P × R) / (P + R)
📨 (a) Case Study 1: Spam Email Detection
Given Data:
TP = 150, FP = 50, TN = 750, FN = 50, Total = 1000
Formulas:
- Accuracy = (TP + TN) ÷ Total
- Precision = TP ÷ (TP + FP)
- Recall = TP ÷ (TP + FN)
- F1 Score = 2 × (Precision × Recall) ÷ (Precision + Recall)
Calculations:
- Accuracy = (150 + 750) ÷ 1000 = 900 ÷ 1000 = 0.90 = 90%
- Precision = 150 ÷ (150 + 50) = 150 ÷ 200 = 0.75 = 75%
- Recall = 150 ÷ (150 + 50) = 150 ÷ 200 = 0.75 = 75%
- F1 = 2 × (0.75 × 0.75) ÷ (0.75 + 0.75) = 1.125 ÷ 1.5 = 0.75 = 75%
✅ Final Answer:
Accuracy = 90%, Precision = 75%, Recall = 75%, F1 = 75%.
💳 (b) Case Study 2: Credit Scoring
Given Data:
TP = 90, FP = 40, TN = 820, FN = 50, Total = 1000
Formulas: (same as above)
Calculations:
- Accuracy = (90 + 820) ÷ 1000 = 910 ÷ 1000 = 0.91 = 91%
- Precision = 90 ÷ (90 + 40) = 90 ÷ 130 = 0.69 = 69%
- Recall = 90 ÷ (90 + 50) = 90 ÷ 140 = 0.64 = 64%
- F1 = 2 × (0.69 × 0.64) ÷ (0.69 + 0.64) = 0.8832 ÷ 1.33 = 0.66 = 66%
✅ Final Answer:
Accuracy = 91%, Precision = 69%, Recall = 64%, F1 = 66%.
💰 (c) Case Study 3: Fraud Detection
Given Data:
TP = 80, FP = 30, TN = 850, FN = 40, Total = 1000
Formulas: (same as above)
Calculations:
- Accuracy = (80 + 850) ÷ 1000 = 930 ÷ 1000 = 0.93 = 93%
- Precision = 80 ÷ (80 + 30) = 80 ÷ 110 = 0.73 = 73%
- Recall = 80 ÷ (80 + 40) = 80 ÷ 120 = 0.67 = 67%
- F1 = 2 × (0.73 × 0.67) ÷ (0.73 + 0.67) = 0.9782 ÷ 1.40 = 0.70 = 70%
✅ Final Answer:
Accuracy = 93%, Precision = 73%, Recall = 67%, F1 = 70%.
🏥 (d) Case Study 4: Medical Diagnosis
Given Data:
TP = 120, FP = 20, TN = 800, FN = 60, Total = 1000
Formulas: (same as above)
Calculations:
- Accuracy = (120 + 800) ÷ 1000 = 920 ÷ 1000 = 0.92 = 92%
- Precision = 120 ÷ (120 + 20) = 120 ÷ 140 = 0.86 = 86%
- Recall = 120 ÷ (120 + 60) = 120 ÷ 180 = 0.67 = 67%
- F1 = 2 × (0.86 × 0.67) ÷ (0.86 + 0.67) = 1.1524 ÷ 1.53 = 0.75 = 75%
✅ Final Answer:
Accuracy = 92%, Precision = 86%, Recall = 67%, F1 = 75%.
📦 (e) Case Study 5: Inventory Management System
Given Data:
TP = 100, FP = 50, TN = 800, FN = 50, Total = 1000
Formulas: (same as above)
Calculations:
- Accuracy = (100 + 800) ÷ 1000 = 900 ÷ 1000 = 0.90 = 90%
- Precision = 100 ÷ (100 + 50) = 100 ÷ 150 = 0.67 = 67%
- Recall = 100 ÷ (100 + 50) = 100 ÷ 150 = 0.67 = 67%
- F1 = 2 × (0.67 × 0.67) ÷ (0.67 + 0.67) = 0.8978 ÷ 1.34 = 0.67 = 67%
✅ Final Answer:
Accuracy = 90%, Precision = 67%, Recall = 67%, F1 = 67%.