Scribd
Upload
102 views

Big Data - An Introduction

The document provides an introduction to big data, defining it as data characterized by high volume, velocity, variety, veracity and value. It discusses three imperatives for big data: talent creation, open data, and open innovation. For talent creation, it outlines the growing demand for data professionals and skills needed. It also discusses universities offering big data courses and professional development programs. For open data, it highlights Malaysia's open data portal and progress. For open innovation, it describes centers of excellence working on impactful big data solutions. The document concludes with opportunities but also needs for awareness, willingness, and funds.

Uploaded by

Mira Mira
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
102 views

Big Data - An Introduction

The document provides an introduction to big data, defining it as data characterized by high volume, velocity, variety, veracity and value. It discusses three imperatives for big data: talent creation, open data, and open innovation. For talent creation, it outlines the growing demand for data professionals and skills needed. It also discusses universities offering big data courses and professional development programs. For open data, it highlights Malaysia's open data portal and progress. For open innovation, it describes centers of excellence working on impactful big data solutions. The document concludes with opportunities but also needs for awareness, willingness, and funds.

Uploaded by

Mira Mira
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 36

BIG DATA:

AN

INTRODUCTION

PROFESSOR DR. R. LOGESWARAN SMIEEE


Big Data Analytics
Seminar 2016
@ APU Data Science
Week 2016
14 Sept 2016

Secretary, IEEE Signal Processing Society


Malaysia Chapter
Dean, School of Postgraduate Studies, APU
(Malaysia)

Outline
Big Data definition
The 3 Imperatives
Talent Creation
Demand, Education, Professional Development

Open Data
Open Innovation
Big Data applications

Some Activities in Big Data


R. Logeswaran

Big Data: An Introduction

Acknowledgement
Malaysia Digital Economy
Corporation Sdn. Bhd., Malaysia
Asia Pacific Centre of
Analytics (APCA),
APU, Malaysia
Other online resources incl.:
EMC, Gartner, Edureka, Active Informatics,
Revolution Analytics, Labour Insight, LinkedIn,
Binary Briyani, IEEE Spectrum, etc.
R. Logeswaran

Big Data: An Introduction

What is
Big Data?
Oxford English Dictionary:
data of a very large size,
typically to the extent that its
manipulation and management present
significant logistical challenges
2011 big data study by McKinsey:
datasets whose size is beyond

the

ability of typical database software tools to


capture, store, manage, and analyze
R. Logeswaran

Big Data: An Introduction

BIG DATA ANALYTICS


Big data is defined by the high

Volume, Velocity, Variety, Veracity and Value


of data generated every day
Growing data

Broadening data

VOLUME

VARIETY

90% of worlds data


generated
over last 2

years

Turning
big data into

Value

80% of the worlds data is

unstructured (text, geospatial,


audio, video)

ECONOMIC
BENEFITS

Establishing the

Increasing data

VELOCITY

GOVERNMENT
BENEFITS

175,000

SOCIETAL
BENEFITS

tweets per
second

R. Logeswaran

Big Data: An Introduction

VERACITY
of big data sources
Big Data technology allows us to
establish quality and accuracy
especially in unstructured data

Big Data
expanding on the 4 fronts
R. Logeswaran

Big Data: An Introduction

Focus @ My
#1Talent

TALENT CREATION

1. Forecasted BDA talent demand Data Professionals

2014

2020 (Malaysia)

4,088

16,000* (incl. of 1,500 Data Scientists)

Source:
IDC

*expected to be revised to 25,000 soon with inclusion of the agricultural sector

2. Universities offering BDA courses

R. Logeswaran

Big Data: An Introduction

3. Professional Development
Game changing 8 week
intense data scientist
acceleration programme.
Top Data Accelerator globally
(Backed by Cornell Uni.)
Massive Open Online Course
(MOOC) with blended
approach - Highest sign up for
Coursera Data Science
MOOC globally for 2015.

R. Logeswaran

Big Data: An Introduction

1. Forecasted BDA talent demand Data Professionals

Skills of a Data
Scientist

Local demand
(Malaysia)

Curious &
Creative
Technical
Quantitative
Skeptical
Communicative
& Collaborative
R. Logeswaran

Big Data: An Introduction

Global Demand for Big Data Jobs


Analytics Jobs

Business Intelligence Jobs

R. Logeswaran

Big Data: An Introduction

Big
Data
Jobs

10

High Salaries & Prospects

R. Logeswaran

Big Data: An Introduction

11

Big Data Jobs


not filled due to
lack of Skilled
Professionals

R. Logeswaran

Big Data: An Introduction

12

Where are Data Science


Professionals needed
Management Consultancy
Retail Sale
Public Administration
Tertiary Education
Banks
Biotechnology R&D
Manufacturing
Insurance
Brokerage
Data Processing & Hosting
R. Logeswaran

Big Data: An Introduction

13

Source: EMC
R. Logeswaran

Big Data: An Introduction

14

R. Logeswaran

Big Data: An Introduction

15

Skills needed by Start-ups in India


Less than 4% of engineers who graduated in
2015 have the skills to be employable in a
technology startup
Top skills required for technology startup-ready
roles, according to the National Employability
Report, include:
technical skills
problem-solving skills
work management and prioritisation
Source:
learning attitude
IEEE Spectrum
Feb 2016
communication skills
R. Logeswaran

Big Data: An Introduction

16

1st University in Malaysia to Offer


Postgraduate Data Science
Programme

2. Universities offering BDA courses

R. Logeswaran

Business,
Statistics, etc.

Big Data: An Introduction

17

3. Professional Development

Expertise in Technologies & Tools

* Basic Flow / Process / Methodology for Data Analytics


R. Logeswaran

Big Data: An Introduction

18

Tools

R. Logeswaran

Tools

Big Data: An Introduction

19

Tools

R. Logeswaran

Big Data: An Introduction

20

Tools

Tools

R. Logeswaran

Big Data: An Introduction

21

Tools

R. Logeswaran

Big Data: An Introduction

22

Tools

R. Logeswaran

Big Data: An Introduction

23

Software Skills for Data Scientist


40

R. Logeswaran

Big Data: An Introduction

24

Dashboard Visualisation

R. Logeswaran

Big Data: An Introduction

25

Focus @ My #2

Open Data

2013

MALAYSIA IS MAKING GOOD PROGRESS


IN OPEN DATA
2014

2020
(Target)

ST

NA

MY

41

UK

US

MY

15,000
Datasets

157,000
Datasets

RD
ST

30

TH

MY

117
Datasets

2014 Open Data Barometer Findings


High in
government
readiness
R. Logeswaran

Low in quality datasets which impacts social


& economic, and therefore low in citizens &
civil society engagement
Big Data: An Introduction

26

Open Data
Malaysia Government Open Data Partoal
(2014): Data.gov.my
United States Open Government Initiative
(2009): Data.gov
United Kingdom (2010): Data.gov.uk
Kenya Open Data Portal (2011)
Ghana Open Data Initiative (2012)
Japan Open Data Initiative (2013)
Others: United Nations, World Bank, EU Open
Data Portal, etc.
R. Logeswaran

Big Data: An Introduction

27

DATA.GOV.MY
Number of Datasets

As at
12/12/15

As at
05/09/16

R. Logeswaran

Big Data: An Introduction

28

Focus @ My #3

Open
Innovation

ACCELERATING INDUSTRY-DRIVEN
COEs FOR IMPACTFUL USE CASES

4 CENTRES OF
EXCELLENCE
formed with MDeC to
create national highimpact
BDA solutions

R. Logeswaran

Some BDA solutions developed:


o Extreme weather projection &
visualization
o Sustainable budget & optimizing
nations financial health
o Dengue hotspot prediction
o Smart manufacturing
o Customer spending behaviour
analysis to increase bank revenue
o Increasing retail revenue
Big Data: An Introduction

29

Flood and Flood Disaster:


Integrated Mobile Solution for
Alert-Search & Rescue (A-SaR)

PRGS
2015

R. Logeswaran

Big Data: An Introduction

30

The Congestion and Accident


Prediction (CAPS)

BIG App
Challenge
2015

- Predicting road congestions on designated


journey.
- Predicting possible accidents at identified
location
- By time
- By weather
- By event
- Proposing alternative routes for safe and fast
journey
Project under discussion for full fledge development
Client: DBKL

R. Logeswaran

Big Data: An Introduction

31

Other Application Opportunities


Education curriculum, job matching, training
Food production, weather, demand, crop cycle
Crime - efficient police deployment, real-time
Energy forecast demand surge, hydroelectric
supply (e.g. based on weather)
Health patients, environment, spread, multimodality examinations, history

and many, many, many more.


R. Logeswaran

Big Data: An Introduction

32

Some Big Data Activities in Malaysia

Big Data Week


Kuala Lumpur

Big Data & Analytics


Innovation Summit, Kuala
Lumpur, 2015

The Open Government


Partnership Seminar & Exhibition

Teradata CTO
Roadshow 2015

R. Logeswaran

Big Data: An Introduction

33

R. Logeswaran

Big Data: An Introduction

34

CONCLUSION
Huge opportunities
Very good job prospects
employment, research, development etc.
Government & industry willingness,
effort and funds required
More open data initiatives required for
community-based applications
Increase activities create more
awareness
R. Logeswaran

Big Data: An Introduction

35

[email protected]

R. Logeswaran

Big Data: An Introduction

36

You might also like