COLLECTION OF DATA, CLASSIFICATION AND TABULATION 3.

1 Introduction: Everybody collects, interprets and uses information, much of it in a numerical or statistical forms in day-to-day life. It is a common practice that people receive large quantities of information everyday through conversations, televisions, computers, the radios, newspapers, posters, notices and instructions. It is just because there is so much information available that people need to be able to absorb, select and reject it. In everyday life, in business and industry, certain statistical information is necessary and it is independent to know where to find it how to collect it. As consequences, everybody has to compare prices and quality before making any decision about what goods to buy. As employees of any firm, people want to compare their salaries and working conditions, promotion opportunities and so on. In time the firms on their part want to control costs and expand their profits. One of the main functions of statistics is to provide information which will help on making decisions. Statistics provides the type of information by providing a description of the present, a profile of the past and an estimate of the future. The following are some of the objectives of collecting statistical information. 1. To describe the methods of collecting primary statistical information. 2. To consider the status involved in carrying out a survey. 3. To analyse the process involved in observation and interpreting. 4. To define and describe sampling. 5. To analyse the basis of sampling. 6. To describe a variety of sampling methods. Statistical investigation is a comprehensive and requires systematic collection of data about some group of people or objects, describing and organizing the data, analyzing the data with 29 the help of different statistical method, summarizing the analysis and using these results for making judgements, decisions and predictions. The validity and accuracy of final judgement is most crucial and depends heavily on how well the data was collected in the first place. The quality of data will greatly affect the conditions and hence at most importance must be given to this process and every possible precautions should be taken to ensure accuracy

while collecting the data. 3.2 Nature of data: It may be noted that different types of data can be collected for different purposes. The data can be collected in connection with time or geographical location or in connection with time and location. The following are the three types of data: 1. Time series data. 2. Spatial data 3. Spacio-temporal data. 3.2.1 Time series data: It is a collection of a set of numerical values, collected over a period of time. The data might have been collected either at regular intervals of time or irregular intervals of time. 3.2.2 Spatial Data: If the data collected is connected with that of a place, then it is termed as spatial data. For example, the data may be 1. Number of runs scored by a batsman in different test matches in a test series at different places 2. District wise rainfall in Tamilnadu 3. Prices of silver in four metropolitan cities 3.2.3 Spacio Temporal Data: If the data collected is connected to the time as well as place then it is known as spacio temporal data. 3.3 Categories of data: Any statistical data can be classified under two categories depending upon the sources utilized. These categories are, 1. Primary data 2. Secondary data 3.3.1 Primary data: Primary data is the one, which is collected by the investigator himself for the purpose of a specific inquiry or study. Such data is original in character and is generated by survey conducted by individuals or research institution or any organisation. The primary data can be collected by the following five methods. 1. Direct personal interviews. 2. Indirect Oral interviews.

3. Information from correspondents. 4. Mailed questionnaire method. 5. Schedules sent through enumerators. 1. Direct personal interviews: The persons from whom informations are collected are known as informants. The investigator personally meets them and asks questions to gather the necessary informations. It is the suitable method for intensive rather than extensive field surveys. It suits best for intensive study of the limited field. Merits: 1. People willingly supply informations because they are approached personally. Hence, more response noticed in this method than in any other method. 2. The collected informations are likely to be uniform and accurate. The investigator is there to clear the doubts of the informants. 3. Supplementary informations on informants personal aspects can be noted. Informations on character and environment may help later to interpret some of the results. 4. Answers for questions about which the informant is likely to be sensitive can be gathered by this method. 5. The wordings in one or more questions can be altered to suit any informant. Explanations may be given in other languages also. Inconvenience and misinterpretations are thereby avoided. Limitations: 1. It is very costly and time consuming. 2. It is very difficult, when the number of persons to be interviewed is large and the persons are spread over a wide area. 3. Personal prejudice and bias are greater under this method. 2. Indirect Oral Interviews: U nder this method the investigator contacts witnesses or neighbours or friends or some other third parties who are capable of supplying the necessary information. This method is preferred if the required information is on addiction or cause of fire or theft or murder etc., If a fire has broken out a certain place, the persons living in neighbourhood and witnesses are likely to give information on the cause of fire. In some cases, police interrogated third parties who are supposed to have knowledge of a theft or a

murder and get some clues. Enquiry committees appointed by governments generally adopt this method and get people s views and all possible details of facts relating to the enquiry. This method is suitable whenever direct sources do not exists or cannot be relied upon or would be unwilling to part with the information. The validity of the results depends upon a few factors, such as the nature of the person whose evidence is being recorded, the ability of the interviewer to draw out information from the third parties by means of appropriate questions and cross examinations, and the number of persons interviewed. For the success of this method one person or one group alone should not be relied upon. 3. Information from correspondents: The investigator appoints local agents or correspondents in different places and compiles the information sent by them. Informations to Newspapers and some departments of Government come by this method. The advantage of this method is that it is cheap and appropriate for extensive investigations. But it may not ensure accurate results because the correspondents are likely to be negligent, prejudiced and biased. This method is adopted in those cases where informations are to be collected periodically from a wide area for a long time. 4. Mailed questionnaire method: Under this method a list of questions is prepared and is sent to all the informants by post. The list of questions is technically called questionnaire. A covering letter accompanying the questionnaire explains the purpose of the investigation and the importance of correct informations and request the informants to fill in the blank spaces provided and to return the form within a specified time. This method is appropriate in those cases where the informants are literates and are spread over a wide area. Merits: 1. It is relatively cheap. 2. It is preferable when the informants are spread over the wide area. Limitations: 1. The greatest limitation is that the informants should be literates who are able to understand and reply the questions. 2. It is possible that some of the persons who receive the questionnaires do not return them. 3. It is difficult to verify the correctness of the informations

furnished by the respondents. With the view of minimizing non-respondents and collecting correct information, the questionnaire should be carefully drafted. There is no hard and fast rule. But the following general principles may be helpful in framing the questionnaire. A covering letter and a self addressed and stamped envelope should accompany the questionnaire. The covering letter should politely point out the purpose of the survey and privilege of the respondent who is one among the few associated with the investigation. It should assure that the informations would be kept confidential and would never be misused. It may promise a copy of the findings or free gifts or concessions etc., Characteristics of a good questionnaire: 1. Number of questions should be minimum. 2. Questions should be in lo 3. Questions should be short and simple. Technical terms and vague expressions capable of different interpretations should be avoided. 4. Questions fetching YES or NO answers are preferable. There may be some multiple choice questions requiring lengthy answers are to be avoided. 5. Personal questions and questions which require memory power and calculations should also be avoided. 6. Question should enable cross check. Deliberate or unconscious mistakes can be detected to an extent. 7. Questions should be carefully framed so as to cover the entire scope of the survey. 8. The wording of the questions should be proper without hurting the feelings or arousing resentment. 9. As far as possible confidential informations should not be sought. 10. Physical appearance should be attractive, sufficient space should be provided for answering each questions. 5. Schedules sent through Enumerators: Under this method enumerators or interviewers take the schedules, meet the informants and filling their replies. Often distinction is made between the schedule and a questionnaire. A schedule is filled by the interviewers in a face-to-face situation with the informant. A questionnaire is filled by the informant which he receives and returns by post. It is suitable for extensive surveys.

Merits: 1. It can be adopted even if the informants are illiterates. 2. Answers for questions of personal and pecuniary nature can be collected. 3. Non-response is minimum as enumerators go personally and contact the informants. 4. The informations collected are reliable. The enumerators can be properly trained for the same. 5. It is most popular methods. Limitations: 1. It is the costliest method. 2. Extensive training is to be given to the enumerators for collecting correct and uniform informations. 3. Interviewing requires experience. Unskilled investigators are likely to fail in their work. Before the actual survey, a pilot survey is conducted. The questionnaire/Schedule is pre-tested in a pilot survey. A few among the people from whom actual information is needed are asked to reply. If they misunderstand a question or find it difficult to answer or do not like its wordings etc., it is to be altered. Further it is to be ensured that every questions fetches the desired answer. Merits and Demerits of primary data: 1. The collection of data by the method of personal survey is possible only if the area covered by the investigator is small. Collection of data by sending the enumerator is bound to be expensive. Care should be taken twice that the enumerator record correct information provided by the informants. 2. Collection of primary data by framing a schedules or distributing and collecting questionnaires by post is less expensive and can be completed in shorter time. 3. Suppose the questions are embarrassing or of complicated nature or the questions probe into personnel affairs of individuals, then the schedules may not be filled with accurate and correct information and hence this method is unsuitable. 4. The information collected for primary data is mere reliable than those collected from the secondary data. 3.3.2 Secondary Data: Secondary data are those data which have been already

collected and analysed by some earlier agency for its own use; and later the same data are used by a different agency. According to W.A.Neiswanger, A primary source is a publication in which the data are published by the same authority which gathered and analysed them. A secondary source is a publication, reporting the data which have been gathered by other authorities and for which others are responsible . Sources of Secondary data: In most of the studies the investigator finds it impracticable to collect first-hand information on all related issues and as such he makes use of the data collected by others. There is a vast amount of published information from which statistical studies may be made and fresh statistics are constantly in a state of production. The sources of secondary data can broadly be classified under two heads: 1. Published sources, and 2. Unpublished sources. 1. Published Sources: The various sources of published data are: 1. Reports and official publications of (i) International bodies such as the International Monetary Fund, International Finance Corporation and United Nations Organisation. (ii) Central and State Governments such as the Report of the Tandon Committee and Pay Commission. 2. Semi-official publication of various local bodies such as Municipal Corporations and District Boards. 3. Private publications-such as the publications of (i) Trade and professional bodies such as the Federation of Indian Chambers of Commerce and Institute of Chartered Accountants. (ii) Financial and economic journals such as Commerce , Capital and Indian Finance . (iii) Annual reports of joint stock companies. (iv) Publications brought out by research agencies, research scholars, etc. It should be noted that the publications mentioned above vary with regard to the periodically of publication. Some are published at regular intervals (yearly, monthly, weekly etc.,) whereas others are ad hoc publications, i.e., with no regularity

about periodicity of publications. Note: A lot of secondary data is available in the internet. We can access it at any time for the further studies. 2. Unpublished Sources All statistical material is not always published. There are various sources of unpublished data such as records maintained by various Government and private offices, studies made by research institutions, scholars, etc. Such sources can also be used where necessary Precautions in the use of Secondary data The following are some of the points that are to be considered in the use of secondary data 1. How the data has been collected and processed 2. The accuracy of the data 3. How far the data has been summarized 4. How comparable the data is with other tabulations 5. How to interpret the data, especially when figures collected for one purpose is used for another Generally speaking, with secondary data, people have to compromise between what they want and what they are able to find. Merits and Demerits of Secondary Data: 1. Secondary data is cheap to obtain. Many government publications are relatively cheap and libraries stock quantities of secondary data produced by the government, by companies and other organisations. 2. Large quantities of secondary data can be got through internet. 3. Much of the secondary data available has been collected for many years and therefore it can be used to plot trends. 4. Secondary data is of value to: - The government help in making decisions and planning future policy. - Business and industry in areas such as marketing, and sales in order to appreciate the general economic and social conditions and to provide information on competitors. - Research organisations by providing social, economical and industrial information.

. FREQUENCY DISTRIBUTION 4.1 Introduction: Frequency distribution is a series when a number of observations with similar or closely related values are put in separate bunches or groups, each group being in order of magnitude in a series. It is simply a table in which the data are grouped into classes and the number of cases which fall in each class are recorded. It shows the frequency of occurrence of different values of a single Phenomenon. A frequency distribution is constructed for three main reasons: 1. To facilitate the analysis of data. 2. To estimate frequencies of the unknown population distribution from the distribution of sample data and 3. To facilitate the computation of various statistical measures 4.2 Raw data: The statistical data collected are generally raw data or ungrouped data. Let us consider the daily wages (in Rs ) of 30 labourers in a factory. 80 70 55 50 60 65 40 30 80 90 75 45 35 65 70 80 82 55 65 80 60 55 38 65 75 85 90 65 45 75 The above figures are nothing but raw or ungrouped data and they are recorded as they occur without any pre consideration. This representation of data does not furnish any useful information and is rather confusing to mind. A better way to express the figures in an ascending or descending order of magnitude and is commonly known as array. But this does not reduce the bulk of the data. The above data when formed into an array is in the following form: 30 35 38 40 45 45 50 55 55 55 60 60 65 65 65 65 65 65 70 70 75 75 75 80 80 80 80 85 90 90 50 The array helps us to see at once the maximum and minimum values. It also gives a rough idea of the distribution of the items over the range . When we have a large number of items, the formation of an array is very difficult, tedious and cumbersome. The Condensation should be directed for better understanding and may be done in two ways, depending on the nature of the data. a) Discrete (or) Ungrouped frequency distribution: In this form of distribution, the frequency refers to discrete

value. Here the data are presented in a way that exact measurement of units are clearly indicated. There are definite difference between the variables of different groups of items. Each class is distinct and separate from the other class. Non-continuity from one class to another class exist. Data as such facts like the number of rooms in a house, the number of companies registered in a country, the number of children in a family, etc. The process of preparing this type of distribution is very simple. We have just to count the number of times a particular value is repeated, which is called the frequency of that class. In order to facilitate counting prepare a column of tallies. In another column, place all possible values of variable from the lowest to the highest. Then put a bar (Vertical line) opposite the particular value to which it relates. To facilitate counting, blocks of five bars are prepared and some space is left in between each block. We finally count the number of bars and get frequency. b) Continuous frequency distribution: In this form of distribution refers to groups of values. This becomes necessary in the case of some variables which can take any fractional value and in which case an exact measurement is not possible. Hence a discrete variable can be presented in the form of a continuous frequency distribution. Wage distribution of 100 employees Weekly wages (Rs) Number of employees 50-100 4 100-150 12 150-200 22 200-250 33 250-300 16 300-350 8 350-400 5 Total 100 4.3 Nature of class: The following are some basic technical terms when a

continuous frequency distribution is formed or data are classified according to class intervals. a) Class limits: T he class limits are the lowest and the highest values that can be included in the class. For example, take the class 30-40. The lowest value of the class is 30 and highest class is 40. The two boundaries of class are known as the lower limits and the upper limit of the class. The lower limit of a class is the value below which there can be no item in the class. The upper limit of a class is the value above which there can be no item to that class. Of the class 60-79, 60 is the lower limit and 79 is the upper limit, i.e. in the case there can be no value which is less than 60 or more than 79. The way in which class limits are stated depends upon the nature of the data. In statistical calculations, lower class limit is denoted by L and upper class limit by U. b) Class Interval: The class interval may be defined as the size of each grouping of data. For example, 50-75, 75-100, 100-125 are class intervals. Each grouping begins with the lower limit of a class interval and ends at the lower limit of the next succeeding class interval c) Width or size of the class interval: The difference between the lower and upper class limits is called Width or size of class interval and is denoted by C . d) Range: The difference between largest and smallest value of the observation is called The Range and is denoted by R ie R = Largest value Smallest value R=L-S e) Mid-value or mid-point: The central point of a class interval is called the mid value or mid-point. It is found out by adding the upper and lower limits of a class and dividing the sum by 2. 53 (i.e.) Midvalue = L U 2 + For example, if the class interval is 20-30 then the mid-value is 20 30 2 + = 25

f) Frequency: Number of observations falling within a particular class interval is called frequency of that class. Let us consider the frequency distribution of weights if persons working in a company. Weight (in kgs) Number of persons 30-40 25 40-50 53 50-60 77 60-70 95 70-80 80 80-90 60 90-100 30 Total 420 In the above example, the class frequency are 25,53,77,95,80,60,30. The total frequency is equal to 420. The total frequency indicate the total number of observations considered in a frequency distribution. g) Number of class intervals: The number of class interval in a frequency is matter of importance. The number of class interval should not be too many. For an ideal frequency distribution, the number of class intervals can vary from 5 to 15. To decide the number of class intervals for the frequency distributive in the whole data, we choose the lowest and the highest of the values. The difference between them will enable us to decide the class intervals. Thus the number of class intervals can be fixed arbitrarily keeping in view the nature of problem under study or it can be 54 decided with the help of Sturges Rule. According to him, the number of classes can be determined by the formula K = 1 + 3. 322 log10 N Where N = Total number of observations log = logarithm of the number K = Number of class intervals. Thus if the number of observation is 10, then the number of class intervals is

K = 1 + 3. 322 log 10 = 4.322 @ 4 If 100 observations are being studied, the number of class interval is K = 1 + 3. 322 log 100 = 7.644 @ 8 and so on. h) Size of the class interval: Since the size of the class interval is inversely proportional to the number of class interval in a given distribution. The approximate value of the size (or width or magnitude) of the class interval C is obtained by using sturges rule as Size of class interval = C = Range Number of class interval = 10 Range 1+3.322 log N Where Range = Largest Value smallest value in the distribution. 4.4 Types of class intervals: There are three methods of classifying the data according to class intervals namely a) Exclusive method b) Inclusive method c) Open-end classes a) Exclusive method: When the class intervals are so fixed that the upper limit of one class is the lower limit of the next class; it is known as the exclusive method of classification. The following data are classified on this basis. It is clear that the exclusive method ensures continuity of data as much as the upper limit of one class is the lower limit of the next class. In the above example, there are so families whose expenditure is between Rs.0 and Rs.4999.99. A family whose expenditure is Rs.5000 would be included in the class interval 5000-10000. This method is widely used in practice. b) Inclusive method: In this method, the overlapping of the class intervals is avoided. Both the lower and upper limits are included in the class interval. This type of classification may be used for a grouped frequency distribution for discrete variable like members in a family, number of workers in a factory etc., where the variable may

take only integral values. It cannot be used with fractional values like age, height, weight etc. This method may be illustrated as follows: Class interval Frequency 5- 9 7 10-14 12 15-19 15 20-29 21 30-34 10 35-39 5 Total 70 Thus to decide whether to use the inclusive method or the exclusive method, it is important to determine whether the variable 56 under observation in a continuous or discrete one. In case of continuous variables, the exclusive method must be used. The inclusive method should be used in case of discrete variable. c) Open end classes: A class limit is missing either at the lower end of the first class interval or at the upper end of the last class interval or both are not specified. The necessity of open end classes arises in a number of practical situations, particularly relating to economic and medical data when there are few very high values or few very low values which are far apart from the majority of observations. The example for the open-end classes as follows : Salary Range No of workers Below 2000 7 2000 4000 5 4000 6000 6 6000 8000 4 8000 and above 3 4.5 Construction of frequency table: Constructing a frequency distribution depends on the nature of the given data. Hence, the following general consideration may be borne in mind for ensuring meaningful classification of data. 1. The number of classes should preferably be between 5 and 20. However there is no rigidity about it. 2. As far as possible one should avoid values of class intervals

as 3,7,11,26.etc. preferably one should have classintervals of either five or multiples of 5 like 10,20,25,100 etc. 3. The starting point i.e the lower limit of the first class, should either be zero or 5 or multiple of 5. 4. To ensure continuity and to get correct class interval we should adopt exclusive method. 5. Wherever possible, it is desirable to use class interval of equal sizes. 4.6 Preparation of frequency table: The premise of data in the form of frequency distribution describes the basic pattern which the data assumes in the mass. Frequency distribution gives a better picture of the pattern of data if the number of items is large. If the identity of the individuals about whom a particular information is taken, is not relevant then the first step of condensation is to divide the observed range of variable into a suitable number of class-intervals and to record the number of observations in each class. Let us consider the weights in kg of 50 college students. 42 62 46 54 41 37 54 44 32 45 47 50 58 49 51 42 46 37 42 39 54 39 51 58 47 64 43 48 49 48 49 61 41 40 58 49 59 57 57 34 56 38 45 52 46 40 63 41 51 41 Here the size of the class interval as per sturges rule is obtained as follows Size of class interval = C = Range 1+3.322 logN = 64 - 32 1+3.322 log(50) = 32 6.64 5 Thus the number of class interval is 7 and size of each class is 5. The required size of each class is 5. The required frequency distribution is prepared using tally marks as given below: Class Interval Tally marks Frequency 30-35 2 35-40 6 40-45 12

45-50 14 50-55 6 55-60 6 60-65 4 Total 50 4.7 Percentage frequency table: The comparison becomes difficult and at times impossible when the total number of items are large and highly different one distribution to other. Under these circumstances percentage frequency distribution facilitates easy comparability. In percentage frequency table, we have to convert the actual frequencies into percentages. The percentages are calculated by using the formula given below: Frequency percentage = Actual Frequency Total Frequency 100 It is also called relative frequency table: An example is given below to construct a percentage frequency table. Marks No. of students Frequency percentage 0-10 3 6 10-20 8 16 20-30 12 24 30-40 17 34 40-50 6 12 50-60 4 8 Total 50 100 4.8 Cumulative frequency table: Cumulative frequency distribution has a running total of the values. It is constructed by adding the frequency of the first class interval to the frequency of the second class interval. Again add that total to the frequency in the third class interval continuing until the final total appearing opposite to the last class interval will be the total of all frequencies. The cumulative frequency may be downward or upward. A downward cumulation results in a list presenting the number of frequencies less than any given amount

as revealed by the lower limit of succeeding class interval and the upward cumulative results in a list presenting the number of frequencies more than and given amount is revealed by the upper limit of a preceding class interval. 4.8.1 Conversion of cumulative frequency to simple Frequency: If we have only cumulative frequency either less than or more than , we can convert it into simple frequencies. For example if we have less than Cumulative frequency, we can convert this to simple frequency by the method given below: Class interval less than Cumulative frequency Simple frequency 15-20 3 3 20-25 10 10 - 3 = 7 25-30 25 25 - 10 = 15 30-35 46 46 - 25 = 21 35-40 58 58 - 46 = 12 40-45 64 64 - 58 = 6 Method of converting more than cumulative frequency to simple frequency is given below. Class interval more than Cumulative frequency Simple frequency 15-20 64 64 - 61 = 3 20-25 61 61 - 54 = 7 25-30 54 54 -39 = 15 30-35 39 39 - 18 = 21 35-40 18 18 - 6 = 12 40-45 6 6 - 0 = 6 62 4.9 Cumulative percentage Frequency table: Instead of cumulative frequency, if cumulative percentages are given, the distribution is called cumulative percentage frequency distribution. We can form this table either by converting the frequencies into percentages and then cumulate it or we can convert the given cumulative frequency into percentages. 4.10 Bivariate frequency distribution: In the previous sections, we described frequency

distribution involving one variable only. Such frequency distributions are called univariate frequency distribution. In many situations simultaneous study of two variables become necessary. For example, we want to classify data relating to the weights are height of a group of individuals, income and expenditure of a group of individuals, age of husbands and wives. The data so classified on the basis of two variables give rise to the so called bivariate frequency distribution and it can be summarized in the form of a table is called bivariate (two-way) frequency table. While preparing a bivariate frequency distribution, the values of each variable are grouped into various classes (not necessarily the same for each variable) . If the data corresponding to one variable, say X is grouped into m classes and the data corresponding to the other variable, say Y is grouped into n classes then the bivariate table will consist of mxn cells. By going through the different pairs of the values, (X,Y) of the variables and using tally marks we can find the frequency of each 63 cell and thus, obtain the bivariate frequency table. The formate of a bivariate frequency table is given below: Format of Bivariate Frequency table x-series Class-Intervals y-series Mid-values Marginal Frequency of Y Class-intervals MidValues fy Marginal frequency of X fx Total fx= fy=N Here f(x,y) is the frequency of the pair (x,y). The frequency distribution of the values of the variables x together with their frequency total (fx) is called the marginal distribution of x and the frequency distribution of the values of the variable Y together with the total frequencies is known as the marginal frequency distribution of Y. The total of the values of manual frequencies is called grand total (N)