Scribd Logo
Upload

Welcome to Scribd!

  • 707 views

    HW3

    Uploaded by

    Arpit Gulati
    This document outlines the requirements for Homework 3 in the Business Data Mining course. It includes 4 problems analyzing datasets using data mining techniques like decision trees and ROC curves. Students must submit a paper of no more than 6 pages by September 23 addressing the questions, which involve tasks like determining misclassification rates, plotting charts, and interpreting evaluation metrics for predictive models.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    HW3

    Uploaded by

    Arpit Gulati
    707 views3 pages
    This document outlines the requirements for Homework 3 in the Business Data Mining course. It includes 4 problems analyzing datasets using data mining techniques like decision trees and ROC curves. Students must submit a paper of no more than 6 pages by September 23 addressing the questions, which involve tasks like determining misclassification rates, plotting charts, and interpreting evaluation metrics for predictive models.

    Original Description:

    sada

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document outlines the requirements for Homework 3 in the Business Data Mining course. It includes 4 problems analyzing datasets using data mining techniques like decision trees and ROC curves. Students must submit a paper of no more than 6 pages by September 23 addressing the questions, which involve tasks like determining misclassification rates, plotting charts, and interpreting evaluation metrics for predictive models.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    707 views3 pages

    HW3

    Uploaded by

    Arpit Gulati
    This document outlines the requirements for Homework 3 in the Business Data Mining course. It includes 4 problems analyzing datasets using data mining techniques like decision trees and ROC curves. Students must submit a paper of no more than 6 pages by September 23 addressing the questions, which involve tasks like determining misclassification rates, plotting charts, and interpreting evaluation metrics for predictive models.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 3

    BUSINESS DATA MINING (IDS 572)

    HOMEWORK 3
    DUE DATE: WEDNESDAY, SEPTEMBER 23 AT 6:00 PM

    Please provide succinct answers to the questions below.


    Your entire write up must be at most six pages long including any figures and/or SPSS printouts.
    You should submit an electronic pdf or word file in blackboard.
    Please include the names of all team-members in your write up and in the name of the file.

    Problem 1. Consider the 100 data points in the file hw3.xls. Each data point is either POSITIVE or
    NEGATIVE. Based on data not shown, two models have been trained to predict the values of these
    data points. For each of the two models, the table gives the probabilities of POSITIVE of each data
    point, the last column shows the actual value.
    (a) Determine the proportion of records p that are POSITIVE.
    (b) Assume the models classify a record as POSITIVE if the probability of POSITIVE is larger
    than 0.5 and otherwise they classify it as NEGATIVE. Determine the misclassification rate for
    each of the two models.
    (c) Again, assume the models classify a record as POSITIVE if the probability of POSITIVE is
    larger than 0.5. Give the coincidence matrices for the two models.
    (d) Plot the cumulative response charts for the two models (you can plot the results for the two
    models in the same chart). Assume a hit is a POSITIVE record.
    (e) Plot the gain charts for the two models (you can plot the results for the two models in the same
    chart). Again, assume a hit is a POSITIVE record.
    Problem 2. Download the file bank-data.csv. The key field in this data set is PEP (Personal Equity
    Plan, a savings product our bank offers). Our goal is to predict whether or not a customer will purchase
    a PEP. We have data from 600 customers as to their purchasing patterns. The fields are
    id
    a unique identification number
    age
    age of customer in years
    sex
    MALE / FEMALE
    region
    inner city/rural/suburban/town
    income
    income of customer
    married
    is the customer married (YES/NO)
    children
    number of children
    car
    does the customer own a car (YES/NO)
    save
    acct does the customer have a saving account (YES/NO)
    current
    acct does the customer have a current account (YES/NO)
    mortgage does the customer have a mortgage (YES/NO)
    pep
    did the customer buy a PEP after the last mailing (YES/NO)
    Use SPSS Modeler to answer the following questions.
    (a) Use 67% of the data set for training and 33% for testing. Create the default C&RT and C5.0
    decision tree models. Use the Analysis node to determine the misclassification rates and
    coincidence matrices for each of the two models on the testing data.
    (b) Use the evaluation node (and relevant charts) to answer the following questions for the decision
    tree model:
    1

    HOMEWORK 3 DUE DATE: WEDNESDAY, SEPTEMBER 23 AT 6:00 PM

    i. What fraction of those who would buy the PEP product do we reach if we mail to only
    half of our customer base?
    ii. If we mail to half of our customer base, what fraction of those would we expect to purchase
    PEP (assuming, of course, we are mailing to a previously unmailed-to group).
    iii. What lift would we get if we mailed to only the most likely 10% of the population?
    (c) Repeat part (b) for the C5.0 model.
    To learn how to draw different graphs using evaluation node in SPSS Modeler, please follow
    the instructions given in the document Evaluation Node on blackboard. This document can
    be found under the SPSS Modeler Documents area.
    Problem 3. A data mining routine has been applied to a transaction dataset and has classified 88
    records as fraudulent (30 correctly so) and 952 as nonfraudulent (920 correctly so).
    (a) Construct the confusion matrix and calculate the error rate, accuracy rate, recall, precision,
    specificity, and false alarm rate. Please include the formulas.
    (b) Consider the decile lift chart below (decile lift chart is the same as lift chart portrayed as a
    decile chart) for the transaction data model applied to a new data.

    Interpret the meaning of the first and second bars from the left.
    (c) Another analyst comments that you could improve the accuracy of the model by classifying
    everything as nonfraudulent. If you do that, what is the error rate?
    (d) Comment on the usefulness, in this situation, of these two metrics of the model performance
    (error rate and lift).

    BUSINESS DATA MINING (IDS 572)

    Problem 4. Suppose we have developed a classifier that will be used in an alarm system. Usually we
    are especially interested in portion of alarms caused by positive events (that should really fire an alarm)
    and portion of alarms caused by negative events. The ratio between positive and negative events can
    vary during time, so we want to measure the quality of our alarm system independently of this ratio.
    The table below shows the results of a probabilistic classifier on a given test set. Draw the ROC curve
    for this classifier (you can use Excel to draw this chart).
    Inst#
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10

    Class
    p
    p
    n
    p
    p
    p
    n
    n
    p
    n

    Score
    0.9
    0.8
    0.7
    0.6
    0.55
    0.54
    0.53
    0.52
    0.51
    0.505

    Inst#
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20

    Class
    p
    n
    p
    p
    n
    n
    p
    n
    p
    n

    Score
    0.4
    0.39
    0.38
    0.37
    0.36
    0.35
    0.34
    0.33
    0.30
    0.1

    You might also like