(@) What is Data? Data refers to raw, unprocessed facts, figures. or symbols that represent information. It can be numbers, text, ——_—_ images, audio, or video. Data is the basic input that needs to be analyzed, processed, or organized to derive meaningful insights. reer Examples of data: * Customer names, ages, and purchase histories ¢ Weather conditions such as temperature and humidity ¢ Social media posts, likes, and sharesData mining data. It involves usin, eniny di i i itterns, relationships, and trends i lata. information can then be used to make data-driven decisions, solve business problems, and uncover hidden insights. Data mining is one of the most useful techniques Ld help_entrepreneurs, re: data. Data mining is also called Knowledge Discovery in Database (KDD), The knowledge discovery process includes Data cleaning, Data integration, Data selection, Data transformation, Data mining, Pattern evaluation, and Knowledge presentation.Main Purpose of Data Mining Statistics Machine tearing Patter recognition Database systems | Visualization Data Mining, Data warehouse Algorithms Information High-performance retrieval Applications ‘computing Basically, Data mining has been integrated with many other techniques from other domains such as statistics, machine learning, ——_—_————— pattern recognition, database and data warehouse systems, information retrieval, visualization, etc. to gather more information tern recogni ata warehouse systems, information ret mo genet more Mirman | about the data and to helps predict hidden patterns, future trends, and behaviors and allows businesses to make decisions. Technically, data mining is the computational process of analyzing data from different perspectives, dimensions, angles and categorizing/summarizing it into meaningful information. Data Mining can be applied to any type of data e.g, Data Warehouses, Transactional Databases, Relational Databases, Multimedia Databases, Spatial Databases, Time-series Databases, World Wide Web.Data Mining as a Whole Process The whole process of Data Mining consists of three main phases: 1. Data Pre-processing - Data cleaning, integration, selection, and transformation takes place 2. Data Extraction — Occurrence of exact data mining 3. Data Evaluation and Presentation - Analyzing and presenting results eee eee eee Data Integration & Cleaning h [Data Selection and] [ ‘Analyzed Data Tranatormation (Resut) Data Evaluation & Data Preprocessing Data Mining a jesentation Applications of Data Mining . Financial Analysis . Biological Analysis . Scientific Analysis . Intrusion Detection . Fraud Detection . Research Analysis ao k WNData Mining is primarily used by organizations with intense consumer demands- Retail, Communication, Financial marketing company, determine price, consumer preferences, product positioning. and impact on sales, customer satstaction. and corporate profits. Data mining enables a retailer to use oint-of-sale records of customer purchases to develop products and promotions that help the organization to attract the customer Benefits of Data Mining 1 Improved decision-making: Data mining can provide valuable insights that can help organizations make better decisions by identifying patterns and trends in large data sets. 2 Increased efficiency: Data mining can automate repetitive and time-consuming tasks, such as data cleaning and preparation, which “an help organizations save time and resources 3. Enhanced competitiveness: Data mining can help organizations gain a competitive edge by uncovering new business opportunities and identifying areas for improvement. 4. Improved customer service: Data mining can help organizations better understand their customers and tailor their products and services to meet their needs. 5. Fraud detection: Data mining can be used to identify fraudulent activities by detecting unusual patterns and anomalies in data. 6. Predictive modeling: Data mining can be used to build models that can predict future events and trends, which can be used to make proactive decisions. 7. New product development: Data mining can be used to identify new product opportunities by analyzing customer purchase patterns and preferences. 8. Risk management; Data mining can be used to identify potential risks by analyzing data on customer behavior, market conditions, ‘nd other factors.What Kinds of Data Can Be mined? * Data mining defines extracting or mining knowledge from huge amounts of data. Data mining enerally use S a huge amount of data is saved and processed. * In Data mining, hidden patterns of i the_multiple categories into a piece ‘ul data. jihis—data_is. = sicing Foond mn creating effective decisions w' ich cl cut value and increase revenue. aos ‘The diagram below shows the data types that can be mined to achieve opumization. ‘An dtustration ofthe types of ata that can be minedt* A flat file is a text_or bil ile with a structure that data mining algorithms can‘easily extract. The dai ed in the flat file has no relationship or path to each oth jat files represent data dictionaries such as CSV files. It is offen used to transrer data to and I ——— Tom servers. * A relational database is a data collection oreanized into tables with rows and columns. The physical schema of a relational database is the le. A relational database i is a schema that defines the relationships between tables. It is mostly used jn applications like Microsoft SQL Server, and Oracle Database. ReD Ms) DataWarchouse ‘A datawarehouse is defined as the collection of data integrated from multiple sources that will queries and decision making, ‘There are three types of datawarehouse: Enterprise datawarehouse, Data Mart and virtual Warehouse. ‘Two approaches can be used to undate data in DataWarehouse: Query-driven Approach and Update-driven Approach. Ne seeraschs ane eee ‘Applieation: Business decision making, Data mining, ete ‘Transactional Databases. ‘Transactional databases isa collection of data organized by time stamps, date, etc to represent transaction in datat ‘Thistype of database has the capablity terol back or undo its operation when a transaction isnot completed or committed. Highly flexible system where users can modify information without changing any sensitive information. ee Nout ahanging ory seme ose Follows ACID property of DBMS. ‘Application: Banking, Distributed systems, Object databases, etc Multimedia Databases ‘Multimedia databases consists audio, video, images and text media. ‘They can be stored on Object-Oriented Databases. ‘They are used to store complex information ina pre-specified formats. ‘Application: Digital ibrares, video-on demand, news-on demand, musical database, etc."Time-Series Database - A time-series database stores sequences of values or events accessed over re measurements of time (e.g., hourly, daily, weekly). An exam includes data collected from the — stock exchange, stock control, and thé measurement of natural phenomena (like temperature and wind).Technologies Used in D i ata rig ba i many techniques from other domain fs ke machine leering, statis, information Feel, ate were ET SEAGR BESTT THE METvormane conputn "soe a seach and deroprent tt Inerdisciptinary nature Is typically very significant. Research 2nd, ‘mining and ts applications prove quite useful ine! Machine Learning: Ithas a main research area that focuses on computer programs that will automatically learn based ‘on the given input data and make intelligent decisions. There are si rrelati ning. For classification and clustering a machine f ning problems that are utligeg in ‘Supervised learning that makes use of class labels to predict information Unsupervised learning doesn’t use class labels similar to clustering but it will discover new classes ee ——_ Semi-supervised learning will redefine the boundaries bet classes and makes use of tae T SSE cba ea et a ee ‘Active learning will ask the user to label the classes. rom unlabeled examples. It will eps Teathng by Seg TSS Roan rele ore Information Retrieval: The technique searches for the information in the document, which may be in text, multimedia, or residing on the Web. It has two main characteristics: + Searched data is unstructured —e—— + QUeries are formed by keywords that don’t have complex structures. The most widely used information retrieval approach is the probabilistic model. Information rettieval combined with data mining techniques is used for finding ‘out any relevant topic in the document or web. utany relevant topic in the document or wed.Statistics: Data mining has an inherent connection with statistics. It studies the collection, and interpretation performs the analysis and helps visualize data presentation. A SoA ee Sg Tor cara masses ang cats modeling. It describes t ehavior of an object in a class and its probability. Statistical models are the mining tasks like classification. fata characteri: . Or we can use the mining task on top of the statistical models. Advantage: Stati del noise, fata values. The tools for forecasting, predicting, or summarizing data can be availed by statistics. Statistics are useful for pattern mining. After mining a classification model, the statistical hypothesis is & ification. A hypothetical test makes the decisions using the test data. The result is statistically significant if it is not likely to have been incurred by chance. Disadvantage: When the statistical medel suse on large doa sett ngeases the cpmmaty ct. When date minirig is used to handle large realtime and streamed data, computation costs increase dramatically.Data visualization * Data visualization is a praphical fepresentation cf quantitative information and data by using visual elements Tike graphs, Charts, and * Data visualization convert large and small data sets into visuals, which is easy to understariO-anit Process Tor humans. * Data visualization tools orovide accessible ways to understand outliers, patterns, and trends in the data. —— Pattern recognition * Pattern recognition is a data analysis method that uses machine learning algorithms to automatically recognize patterns and regularities n data. This data-can-be-anvihing from text and images 10 sounds or other definable qualities. Pattern recognition systems can recognize familiar patterns quickly and accurately.* 2 Palablarchouse data: Collection 4 data ‘integrated from AtPhement Sauces wibs querying and decteion making cn daka To data werehouce, data te stoned In mrullideimencfora) Shucherre (datzacee) where each Airnendfon 6 cach attributecon GD Source - 1 on 5 alu sis P — Pat GD Oe -2 Dolan Source -3uses of datamining @ Data mining is a process used to extract valuable insights and patterns from large datasets. It ‘employs statistical, mathematical, and machine leaming techniques to analyze and interpret data, ‘Some key uses of data mining include: 1. Customer Segmentation: © Data mining can help businesses group customers based on behaviors, pi demographics, enabling more personalized marketing. 2. Fraud Detection © Financial institutions use data mining to detect anomalies or unusual patterns that may Score nations —eeEEeee—E indicate fraudulent activity, 3. Market Basket Analysis: © This technuque analyzes customer purchase behavior to determine products often bought together. helping businesses optimize product placement and ctoss-sell 4. Predictive Analytics: + © Companies can predict future tends, customer behaviot, of stock prices by analyzing @ Message ChatGPT a sen &uevp@uas -eOx- 4. Predictive Analytics: © Companies can predict future trends, customer behavior, of stock prices: i panies can eet re end cnet beh res by nang hustoncal data. 5. Recommendation Systems: © E-commerce and streaming platforms use data mining to suggest products movies, or music based on user preferences and past behaviors (e.g. Amazon, Netflix) 6 Healthcare * Data mining is used to identiy disease tends. predict patient outcomes, and optimize treatment plans by analyzing patient records. 7. Risk Management: © In financial services and insurance, data mining helps assess risk by historical data ancial services and inewrance isk by analysing historical dat ond denoting key factors. 8. Customer Retention: ‘© By analyzing customer interactions and feedback, businesses can entity at-nsk customers. and take steps to retain themete ee ne ee te ae re a & Network Optimization: Analyzing usage patterns to improve network performance, Fraud Prevention: Detecting suspicious call patterns or unauthorized use. 6 Manufacturing Predictive Maintenance: Monitoring equipment performance to predict and prevent failures, Quality Control: Analyzing production data to identify defects and improve product quality. ‘Supply Chain Optimization: Enhancing logistics and reducing costs by analyzing supply chain data. de Education tt Performance Prediction: Identifying students at risk of underperforming. Personalized Learning: Recommending talored content and eSTIRg pat Tors ucents Curriculum Design: Analyzing data to develop effective course structures. 8. Government and ic Ir Cram Preaictiorrand Breve Analyzing crime data to identify high-risk areas and_ prevent incidents. Public ith Analysis: Monitoring disease outbreaks and predicting epidemic trends. Tax Fraud Detection: Identifying anomalies in tax filings to detect evasion. 9. Transportation Traffic Management: Analyzing traffic patterns to optimize road usage and reduce congestion. Route Optimization: Improving delivery efficiency by identifying the best routes. Predictive Maintenance: Monitoring vehicle data to schedule timely maintenance.jor aMining2— meee Te Gen ten inner | bLues related ‘aa cig which ose whon databee ow incomplata » rolig & conrain of ming valu. * Gane ae. oer Wiures os well Had crite duo b Inoupiient é inappropridla information Pent in Ho data. bases - Po Iauasore os dawipied ay Ui) Umikd 4 Inelovant Information Scanned by CamScanner ti) Moiuy 4 miming dala. iN Gi) Hamon inteadtion 4 prior wero ledge * tS Ci longa dao. se & high dinaraiorolty tv) un entainity be ™) dyromic ups Major Issues in Data Mining 1, Data Quality Quality of data is paramount in data mining. issues such os nolsy data incomplete data, ond unstructured data can significantly impair_the effectiveness of genuine data mining measure ond efforts. Data must be ‘accurate, complete, and consistent for genuine data mining measures to succeed, Solutions: += Implement robust data cleaning methods to preprocess data and Improve quality. ‘+ Use data mining tools that can handle heterogeneous data sources and diverse data types. * Continuously monitor and update data quality standards, 2. Data Privacy and Security ‘As data mining often involves analyzing sensitive information, ensuring data privacy and security is crucial. Unauthorized access to data can lead to privacy ‘and data security breaches and misuse of Solutions: + Employ encryption and access controls to protect data. + Develop policies and frameworks that adhere to data privacy regulations. + Utilze data mining techniques that minimize privacy risks, such as [gnonymnizution ond diflerential privany.clues] Complex and Diverse Data Types * Structured vs. Unstructured Data: Dealing with different formats (e.g., text, images, audio). een eel * High-Dimensional Data: Managing data with ial lanl -1obH= 1A a olO L(y ee * Spatial and Temporal Data: Analyzing data related to locations and time. paicnainanaieae Rie ke Scalability and Efficiency SED * Large Datasets: Processing massive amounts of data. * Computational Complexity: Ensuring algorithms are efficient for large-scale analysis. * Distributed Computing: Utilizing multiple 2, Sy aS a PT eae . machines for parallel processing.3. Handling Complex and Diverse Data Types Data mining is often challenged by the need to process complex types of data such as spatial data, temporal data, and media data. These data types require specialized approaches to analyze and extract useful insights. Solutions: * Leverage complex data perception methods tailored to specitic data types. * Use parallel data mining algorithms and distributed data mining algorithms to handle large-scale and complex datasets. * Develop flexible data mining tools that can adapt to various data formats and structures. 4. Scalability and Efficiency With the growth of data volume, scalability and efficiency refining data mining requests become. s, Data miners must be able to handle enormous ata miners must be able to handle enormous data sets without compromising performance,5. Integration with Heterogeneous Data Sources Data_often_comes_from multiple sources _with varying formats, making integration into data warehouses a significant challenge. Cambining data from different systems requires careful handling to ensure consistenc . ‘Solutions: * Use data integration tools to combine data trom different sources into a unified data archive. * Implement £1\ (Extract, Transform, Load) processes to standardize data from heterogeneous sources. + Develop metadata systems to manage and align data across different platforms. 6. Interpretation and Usability of Results The ultimate goal of a data mining algorithm is to provide actionable insights. However, interpreti ‘ensuring their usability can be difficult, especially for non-technical stakeholders. —Saa=a—T——_eeowwE—>nnam—uauaw—mwszn—_] ‘Solutions: + Enhance data visualization capabilities to make results more accessible and understandable. + Provide training and support for users to int insights. * Develop interactive systems that allow users to explore and refine data mining outputs. pret and act on data mining 7. Dynamic and Changing Data In many applications, data is dynamic and constantly changing, This poses a el ining techniques that must adapt to new information and evolving patterns input data. ‘Solutions: + Implement real-time data mining systems that can update and adapt to new data on-the-fly * Use ad hoc data mining approaches to handle unplanned and dynamic data queries. * Develop data mining approaches that can learn incrementally and adjust to changing data landscapes. 8. Legal and Ethical Concerns Data_mining can raise significant legal_and ethical issues, particularly concerning the use of personal data. Ensuring compliance with regulations and addressing ethical considerations Gre crucial.Different Types of Data in Data Mining Here are Different Types of Data in Data Mining : 1. Structured Data Structured data is highly organized and easily searchable. It resides in fixed fields within records or files, typically in relational databases or spreadsheets. Examples include SQL databases and Excel spreadsheets. a Mate eS 2. Unstructured Data Unstructured data lacks a predefined format or structure, making it more complex to analyze. It is often textual or multimedia content, Examples include emails, social media posts, images, videos, and audio files. 3. Semi-Structured Data Semi-structured data doés not conform toa rigid structure but contains tags or markers to separate data elements. It is a middle ground between structured and unstructured data. Examples include XML files and JSON documents. 4. Time-Series Data Time-series data is a ints collected or recorded at specific ti intervals. It is crucial for analyzing trends and patterns over time. Examples include stock prices, weather data, and sensor readings.5. Spatial Data Spatial data represents the physical location and shape of objects in geographic spare. Tt is used in geographic information systems (GIS). Examples include maps, satellite images, and location-based services data. 6. Graph Data Graph data re, ionships between e ith nodes (entities) and edges (relationships). It is used in network analysis. Examples include social networks, citation networks, and communication nefyerks. 7. Text Data Text data comprises written words, sentences, and paragraphs. It is abundant and used in natural language processing (NLP). Examples include emails, reports, and web pages. 8. Multimedia Data Multimedia data includes a combination of text, audio, images, and video. It is complex and requires specialized tools for analysis. Examples include podcasts, movies, and photo collections. Conclusion Understanding the different types of data in data_mining is crucial for selecting the appropriate tools and techniques for analysis. Structured data is highly organizedMice k ernie rene RuLe bes Soa Rus Cur) cus Ea es ec RR from errors, inconsistencies, and inaccuracies, making it suitable for reliable decision-maki Data quality encompasses various aspects, including correctness, timeliness, relevance, and adherence to predefined standards. Organizations prioritize data quality to ensure that their information assets meet the required standards and contribute effectively to business processes and decision-making. Effective data quality Ee management invelves processes such as data profiling, cleansing, validation, and monitoring to maintain and Ee et = ee Reus aac ‘* Discover: Use data profiles to comprehend cause abnormalities. ‘+ Define: Specify the standards for standardization and cleaning. pe SNe ee Soe ee cee gee a Oe Cocca ae enaWhy is Data Quality Critical in Data Mining? 1.Reliable_Insights: High-exceptional facts result in greater accurate evaluation and dependable insights. Only accurate or complete information can bring about correct conclusions and unreliable patterns, impacting choice-making tactics. 2.Enhanced Decision Making: Quality records form the idea for making knowledgeable selections. Organizations closely depend on facts-driven insights to strategize, forecast tendencies, and identify opportunities or risks. Poor-first-class records can misguide choice-makers, leading to luxurious mistakes. 3. Cost and Efficiency: Poor high-quality records may be pricey for businesses. It frequently results in inefficient approaches, as valuable time and resources are spent on cleansing and rectifying inaccurate records instead of on meaningful evaluation. 4, Customer Trust and Satisfaction: In industries like finance, healthcare, and retail, IN which record accuracy Is crucial, retaining amazing data builds consideration amongst clients. Inaccurate records about clients can damage relationships and erode agreements.Strategies to Improve Data Quality - Data Quality Assessment: Conducting everyday assessments to evaluate records first-rate in opposition to predefined metrics is critical. It helps in figuring out regions that need improvement and setting up benchmarks for the best. 2.Data Standardization_and_ Metadata Management: Implementing requirements for facts codecs, naming conventions, and metadata control guarantees consistency across datasets. This simplifies statistics integration and complements statistics best. 3. Data_Enrichment_and Validation: Augmenting datasets with additional applicable records and validating incoming records at the point of access can decorate accuracy and completeness. 4.Training and Awareness: Educating personnel approximately the significance of high-quality facts and providing education on information entry, protection, and governance techniques can substantially make contributions to preserving high standards. a Collaboration and Stakeholder Involvement: Involving numerous stakeholders, such as statistics analysts, records scientists, IT specialists, and business customers, fosters a collaborative approach towards making sure statistics are excellent at every degree of the data lifecycle. 6.Continuous Improvement: Establishing a subculture of continuous improvement with the aid of mastering from beyond information pleasant problems, refining processes, and adopting rising technologies or great practices facilitates evolving statistics nice control strategies. 7. Data_Quality Tools and Technologies: Leveraging records, pleasant tools, and technology, which include information profiling tools, facts cleansing software programs, and information exceptional dashboards, streamlines the identity and backbone of information issues.Challenges in Data Quali allenges in Data Quality Cee Cu acer gaa incomplete Data: It can be difficult to collect thorough and accurate data, resulting in gaps and flaws in the Crone uni ER ce + Issues with Data Accuracy: Inconsistencies in data sources, faults during data input, and system Caine nese es Cgc OMe eae ced ee eka) eRe a CU agen Cure ecu Le ae ere od Concerns, Data Quality Monitoring, Technology Limitations, and human error. Benefits of Data Quality Bucur Acie one a Bee et oun eau aaa ue TR ne eee Cee eek ce ate SRO CEL Eee et erreur no Cee ca) oc OD ur ke ue ae ae eC a Ck a) _—$—$—<<— Pees Seen Eee nT eRe ue ee ea Rcd Crees ‘+ Cost saving: Good data reduces the need for revisions, which saves money. pa ‘+ Credibility and Trust: Credibility and confidence are gained by organizations with high-quality data among acre2+ KD [anatase pias in Databose) i Dalamuini ing also known O3 hrowlacee discoury in : » databases, reas % the. nontalval ‘axkraction of- im ‘ety . Priviowly unknown and potty. welul information #70n ¥ dala. ldred ip difubase-— . » oer : Shapes’ of KDDpTo@s :— > pe Ghere are. + -diyerert 3+ ; Jo Koo pros. his pmax row ddla os Up 4 provides 5 Welw information dasined by wrexs ax tho op. > * Jo objectua of KP) procs is -b Hain” at qood ; mle about Ha Spon organisation's ) 7 , ) ) ) Suryourdi % flo 91000 Ommunicas volth domain experts $ _ tecbnlal. eapert eine He aauraey »toilany “ae Ge‘Ta follacing one Yo aged in KOP Pract w) Dale. thong 4 prepretatie Hoge O) Doda. i ration : Citi) Dale, Selection stage + & Data “hanstormation ¢ reduction ahoge &) pate Mining adlscous : Wi) Rattan ion pretation 4 analysis Stage rPPeeognggangagy ~ Wil) Hrodladge vuualizesion age ti Delany 4 peeproening Hage t= . Dale. chanting ham Prec@n of Yemoving_u ings dat trom © Ha. databases » Qe main Purpose, of preprowuig bb inprore Ape swality OF the, dab by tiling. out wining valu, 2 conjiqering ddle t mam: sme Hat it I in caaltont * -_-—-_—¢! : tora? - toy e! til) Dale. itlogration “~ To this sage,’ numerous dace S007@3 are tonbirad lig) oe ar iol ~base . : tii) Dada. see tion stage '~ Dada. wchidh by veouived for alata mining prowe tan be ediacid for malipl $ = helarog eno us dafr Sovres buch a3 dabrbare,, file tom © elethonic Souras. data seluction bo pweox shore ty ‘ a , PRRppr0 priate, dila reouied tor analysis are $elchod trom ‘YWa_databese-’ ~ UY Dette. FiansPormation 4. Fadkiction stage : - Ip she anst- » -arbion sags dso ie tautheN R28 dala, Gountas 61 tonvertid ih ao je appropriate format tor data mining ows b form ine regadion function « 2 also bo vequired thal ome dab. should be onuritd ints) wore, wobt toned path Pe vedwtion_b wed 4 dolrente Wo ne~ oP por iblo val od oP ata chich are being Corbidered wwetthou apecting ie [» (VY) DalaMining disorer Hage - hoses receniaen ; 2b on ingot OO [? bicep dato. patton tor evaluation 7 db dt 1S eg at ter relations 7) patioras . ot es ential usepel rouslesge Knowsleda2, frbm, mrassi Le, ae eata of data, “stored in Hories . ivi) Patter Interpretation % — > In Ho, Hage, pattern, qenerabed 93 lp tron chia wie Haga are hantouned in Krowsledee which & used in cciion supnt axslom, a telps_ make dedhion qeganc! Ing_boo ‘he. oe areb be proand b te urer- goth chlanirl 4 An =< To parton vepeatid yy ax analysis Sap. cependont a — tlar kind o& = aloorittm, fddd ‘ed eee 3 a ere eeewi) Kreualedg viualigakon “Hage 7 is soot os Vibualization 4 prerentation motbody ore used -b srepresent te 6. os obtained from mei « mentation of wna be indw, fom notte, Oe treo bron ort oe ele. presentation ao doals usith storing wefad information ins Knoalasge- base Jor inlrwiedeting wie. Sone ot H2_ wis uabzation tach nial 42s inchucts, geomatrie, icon based, pital e based etc. using visual: zohon , uyer ean Sunmatize , etacde- c rap more civicellk result In easy undowtardable (frnalasy) Mauro 7— Gups of KOD pani 999 MONA ODHOMHOHHHGTLHN Scanned by CamScanner