Chapter 1: Introduction

Question 1

New Sources of Data

Answer 1

• Tweets 12tb
• Facebook 25tb
• Google, youtube …
• RFID
• Smart Meters
• Cameras
• GPS

Question 2

1 Source of large data

Answer 2

• Customer transactional data –> how do customers behave?

Question 3

Traditional Data Warehousing

Answer 3

• Several Sources (e.g. online transaction system) –>
• Extractor / Monitor –>
• Integration System (<–> Meta Data) –>
• Data warehouse (Mngmt decision support)
• –> Clients

Question 4

Volume, Velocity, and Variety

Answer 4

• Volume: Enterprises are awash with ever-growing data of all types.
  
  • Turn 12 terabytes of Tweets each day into improved product sentiment analysis
  • Convert 350 billion annual meter readings to better predict power consumption
• Velocity: For time-sensitive processes such as catching fraud, big data must be used as it streams into your enterprise in order to maximize its value.
  
  • Scrutinize 5 million trade events created each day to identify potential fraud
  • Analyze 500 million daily call detail records in real-time to predict customer churn
    faster
• Variety: Big data is any type of data - structured and unstructured data such as text, sensor data, audio, video, click streams, log files and more.
  
  • Monitor 100’s of live video feeds from surveillance cameras to target points of interest
  • Exploit the 80% data growth in images, video and documents to improve customer satisfaction

Question 5

Aggregating Data from Different Sources

Answer 5

The challenge for most organizations is to manage and analyze the various sources of structured, structured, and streaming data.

• Websites
• Billing, ERP, CRM
• RFID
• Network switches
• Social media

Question 6

New Trends in Data Organization

Answer 6

• Main memory databases are able to run queries in seconds (which took hours!)
• Distributed file systems allow for effective parallelization (e.g., Apache Hadoop)

Question 7

Business Analytics (Definition)

Answer 7

Business analytics makes extensive use of statistical analysis, including explanatory and predictive modeling, and fact-based management to drive decision making. It is therefore closely related to management science. Analytics may be used as input for human decisions or may drive fully automated decisions.

Question 8

Descriptive Analytics

Answer 8

What has occurred?

How much did I sell?

BI, Data engineering, statistics …

Data Engineering and Statistics:

Organize data, execute large queries, describe means, trends, and test hypotheses

Question 9

Predictive Analytics

Answer 9

What will occur?

Try to understand behaviour. E.g. switching customers

Data Mining and Econometrics

Forecast events, predict time series, or discrete choice decisions of customers

Question 10

Prescriptive Analytics

Answer 10

What should occur?

Network flow, Management science …

Algorithms and Optimization

Develop algorithms and optimization models for planning, scheduling,

pricing, and revenue mgt.

Question 11

Relationship to Business Intelligence (BA related to predictive / inductive statistics and BI related to descriptive analytics / statistics)

Answer 11

• Business analytics (related to predictive analytics / inductive statistics)
  
  • focuses on developing new insights and understanding of business

performance based on data and statistical methods.

* may be used as input for human decisions or may drive fully automated decisions. * Business intelligence (related to descriptive analytics / statistics)
* traditionally focuses on using a consistent set of metrics to both measure past performance and guide business planning, which is also based on data and statistical methods.
* is often associated with querying, reporting, OLAP, and "alerts".

Question 12

From Data to Information (Flow)

Answer 12

• Data consolidation (Data input and Querys) –> DWH
• Selection and processing (make sense out of large table)
• Business analytics (model that fits data)
• Interpretation and evaluation (insights)

Question 13

Predictive Analytics

Answer 13

• Algorithms and Databases
  
  • Association Rule Algorithms
  • Algorithm Design Techniques
  • Algorithm Analysis
  • Statistics and Econometrics
• Statistics and Econometrics
  
  • Bayes Theorem
  • Regression Analysis
  • EM Algorithm
  • Clustering
  • Time Series Analysis
• Machine Learning and Data Mining
  
  • Decision Tree and other Classification Algorithms
  • Clustering
  • Neural Networks

Question 14

Numerical prediction

Answer 14

Given a collection of data with known numeric outputs, create a function that outputs a predicted value from a new set of inputs.

E.g. Given gestation time of an animal, predict its maximum life span.

Question 15

Classification

Answer 15

• From data with known labels, create a classifier that determines which label to apply to a new observation
• E.g. Identify new loan applicants as low, medium, or high risk based on existing applicant behavior.

Question 16

Clustering

Answer 16

• Identify “natural” groupings in data
• Unsupervised learning, no predefined groups
• E.g. Identify clusters of “similar” customers.

Difference to classification: you do not know the groups

Question 17

Association Rule Analysis

Answer 17

• Identify relationships in data from co-occuring terms or items.
• E.g., analyze grocery store purchases to identify items most commonly purchased together.

Market basket analysis (milk, sugar, eggs)

Question 18

What is a Model?

Answer 18

Mathematical functions

• Mathematical combination of attribute values
• E.g. linear model, non-linear model
• CPU performance prediction

E.g.:

• Decision tree:
  
  • Study: >= 10hrs –> Do homework <10 hours test well…
• Neural networks:

Question 19

Model selection

Answer 19

• Build model
• evaluate performance
• meet criteria? no –> build model
• yes: interpret model

Question 20

Most important algorithms

Answer 20

• Regression
• Decision tree
• Cluster analysis

Question 21

Examples of Analytics in Retailing

Answer 21

• Campaign management
• Product recommendations
• Customer profitability analysis
• Customer segmentation analysis
• Pricing products
• Forecasting revenues
• Analysis of clickstream data

Question 22

CRM Marketing and examples

Answer 22

• CRM marketing is #1 area to which data mining is applied.
• Reccomender systems: (systems for recommending items) Amazon, netflix …
  
  • increase sales
  • Customer A buys
  • Customer B searches what A bought, gets presented what he also bought…
• Collaborative Filtering
  • Maintain a database of many users’ ratings of a variety of items. For a given user, find other similar users whose ratings strongly correlate with the current user.
  • Recommend items rated highly by these similar users, but not rated by the current user.