This document discusses various methods for evaluating operating system performance, including timings, microbenchmarks, traces, profiles, and kernel programs. It provides examples of common performance measures like response time, throughput, and utilization. It also explains how performance evaluation benefits different groups and how operating systems can improve performance through techniques like efficient scheduling policies.
Download as DOCX, PDF, TXT or read online on Scribd
0 ratings0% found this document useful (0 votes)
159 views
Week 6 CH 16 Distributed Processors
This document discusses various methods for evaluating operating system performance, including timings, microbenchmarks, traces, profiles, and kernel programs. It provides examples of common performance measures like response time, throughput, and utilization. It also explains how performance evaluation benefits different groups and how operating systems can improve performance through techniques like efficient scheduling policies.
Department of Computer Science & Information Technology 1
Advance Operating Systems Assignment #6 Of Chapter 16 Eighth Edition of Operating System Concepts by Avi Silberschatz Peter Baer Galvin & Greg Gagne 2009
Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 2
[email protected] Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 3
1. How do you suppose performance evaluation benefits consumers, developers and users? 2. What resource of a computer system probably has the greatest impact on performance? Ans: 1) Performance evaluation provides consumers with a basis of comparison to decide between different systems, developers with useful information on how to write software to efficiently use system components and users with information that can facilitate tuning the system to meet the requirements of a specific setting. 2) The processor(s). 1. How has the focus of performance studies shifted over the years? 2. Give several examples of how operating systems can improve performance, as discussed in the preceding chapters. Ans: 1) In the early years, hardware was the dominant cost, so performance studies focused on hardware issues. Today, it is recognized that sophisticated software can have a substantial effect on performance. 2) Operating systems can improve disk and processor scheduling policies, implement more efficient thread-synchronization protocols, more effectively manage file systems, perform context switches more efficiently, improve memory management algorithms,etc. System Tuning: When the system is installed at the user's site, both the vendor and the user seek to obtain optimal performance. Administrators fine-tune the system to run at its best in the user's operating environment. This process, called system tuning, can often cause dramatic performance improvements, once the system is adjusted to the idiosyncrasies of the user installation. 1. When would an evaluator use performance projection in preference to selection evaluation? 2. How does performance monitoring facilitate system tuning? Ans: 1) Selection evaluation help an evaluator choose among existing systems. Performance projection helps the evaluator predict the performance of systems that do not exist yetor how anticipated modifications to existing systems would perform. 2) An evaluator can monitor performance to determine which modifications would most likely increase system performance. Some common performance measures are: Turnaround timeThis is the time from submission of a job until the system returns a result to the user. Response timeThis is an interactive system's turnaround time, often defined as the time from a user's pressing an Enter key or clicking a mouse until the system displays its response. System reaction timeIn an interactive system, this is often defined as the time from a user's pressing Enter or clicking a mouse until the first time slice of service is given to that user's request.
A certain mean value can be produced by averaging a series of identical or nearly identical values, or it can be produced by averaging a wide variety of values, some much larger and some much smaller than the calculated mean. Therefore, other performance measures often employed are: Variance in response times (or of any of the random variables we discuss)The variance of response times is a measure of dispersion. A small variance indicates that the response times experienced by users are generally close to the mean. A large variance indicates that some users are experiencing response times that differ widely from the mean. Some users could be receiving fast service, while others could be experiencing long delays. Thus the variance of response times is a measure of predictability; this can be an important performance measure for users of interactive systems. ThroughputThis is the work-per-unit-time performance measurement. WorkloadThis is the measure of the amount of work that has been submitted to the system. Often, evaluators define an acceptable level of performance for the typical workload of a computing environment. The system is evaluated compared to the acceptable level. CapacityThis is a measure of the maximum throughput a system can attain, assuming that whenever the system is ready to accept more jobs,another job is immediately available. Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 4
UtilizationThis is the fraction of time that a resource is in use. Utilization can be a deceptive measure. Although a high-percent utilization seems desirable, it might be the result of inefficient usage. One way to achieve high processor utilization, for example, is to run a process that is in an infinite loop!. "poor utilization" is actually a positive measure in certain kinds of systems, such as hard real-time systems, where the system resources must stand ready to respond immediately to incoming tasks, or lives could be at risk. Such systems focus on immediate response rather than resource utilization. 1. What is the difference between response time and system reaction time? 2. (T/F) When a processor spends most of its time in user mode, the system achieves efficient processor utilization. Ans: 1) Response time is the time required for the system to finish responding to a user request (from the time the request is submitted); system reaction time is the time required for the system to begin responding to a user request (from the time the request is submitted).2) False. If the processor is executing an infinite loop, this system is not using the processor efficiently. When the performance of a system must be evaluated in a given environment, designers often use trace data. A trace is a record of system activity typically a log of user and application requests to the operating system. Another method for capturing a computing system's execution environment is profiling. Profiles record system activity while executing in kernel mode, which can include operations such as process scheduling, memory management and I/O management. For example, a profile might record which kernel operations are performed most often. 1. How do evaluators use trace data? 2. Explain the difference between traces and profiles. Ans: 1) Trace data permits evaluators to compare the performance of many different systems that operate in the same computing environment. Trace data describes this environment so that evaluators can obtain performance results relevant to the systems' intended use 2) Traces record user requests, whereas profiles log all activity in kernel mode. Therefore traces describe a computing environment by capturing the user demand for particular kernel services (without regard to the underlying system), and profiles capture operating system activity in a given environment. 1. How can results from timings and microbenchmarks be misleading? How are they useful? 2. Which performance measure can be combined with microbenchmarks to evaluate operating system performance? Ans: 1) Microbenchmarks measure the time required to perform specific primitives (e.g.. process creation), and timings perform quick comparisons of hardware operations (e.g., add instructions). Neither of these measurements reflects system performance as a whole. However, microbenchmarks and timings can be useful in pinpointing potential areas of inefficiency and evaluating the effect of small modifications on system performance. 2) Profiles, which record how often an operating system primitive is used, can be combined with microbenchmarks to evaluate operating system performance. A kernel program is another application-specific performance evaluation tool, although it is not often used. A kernel program can range from an entire program that is typical of one executed at an installation or a simple algorithm such as a matrix inversion. 1. What is a benefit of application-specific evaluation? What is a drawback? 2. Why do you suppose kernel programs are rarely used? Ans: 1) Application-specific evaluation is useful in determining whether a system will perform well in executing the particular programs at a given installation. A drawback is that a system must be evaluated by each installation that is considering using it; the system's designers cannot simply publish one set of performance results. 2) Kernel programs require time and effort to prepare and time. Also, they are "executed on paper" and can introduce human error. Often, it is easier to run the actual program, or one similar to it, on the actual system than calculate the execution time for a kernel program. Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 5
1. Explain the relative merits of complex and simple analytic models. 2. What are some benefits of using analytic modeling? Ans: 1) Complex analytic models are more accurate, but it might not be possible to find a mathematical solution to model a system's behavior. It is easier to represent the behavior of a system with a simpler model, but it might not accurately represent the system. 2) There is a large body of results that evaluators can draw from when creating models; analytic models can provide accurate and quick performance results; and they can be modified relatively easily when the system changes. A benchmark is a program that is executed to evaluate a machine. Commonly, a benchmark is a production program which is typical of many jobs at the installation. 1. How can benchmarks be used to anticipate the effect of proposed changes to a system? 2. Why are there no universally accepted "standard" benchmarks? Ans: 1) In general, benchmarks are effective only for determining the results after a change or for performance comparisons between systems, not for anticipating the effect of proposed changes to a system. However, if the proposed changes match the configuration of an existing system, running the benchmark on that system would be useful. 2) Benchmarks are real programs that are run on real machines, but each machine might contain a different set of hardware that runs a different mix of programs. Therefore, benchmark designers provide "typical" application mixes that are updated regularly to better approximate particular environments. Synthetic programs (also called synthetic benchmarks) are similar to benchmarks, except that they focus on a specific component of the system, such as the I/O subsystem and memory subsystem. 1. Explain why synthetic programs are useful for development environments. 2. Should synthetic programs alone be used for a performance evaluation? Why? Ans: 1) Synthetic programs can be written fairly quickly and can test specific features for correctness. 2) No, synthetic programs are "artificial" programs used to test specific components or to characterize a large set of programs (but not one in particular). Therefore, although producing valuable results, they do not necessarily describe how the entire system will perform when executing real programs. It is usually a good idea to use a variety of performance evaluation techniques. Simulation is a technique in which an evaluator develops a computerized model of the system being evaluated. The evaluator tests the model, which presumably reflects the target system, to infer performance data about this system. Simulators are generally of two types: Event-driven simulatorsThese are controlled by events that are made to occur in the simulator according to probability distributions. Script-driven simulatorsThese are controlled by data carefully manipulated to reflect the anticipated environment of the simulated system; evaluators derive this data from empirical observations. 1. Which produces more consistent results, event-driven or script-driven simulators? 2. (T/F) Complex simulators are always more effective than simpler ones. Ans: 1) Script-driven simulators produce nearly the same result each run because the system always uses the same inputs. Because event-driven simulators dynamically generate input based on probabilities, the results are less consistent. 2) False. Although complex simulators attempt to model a system more completely, they still might not model it accurately. 1. Why do software performance monitors influence a system more than hardware performance monitors? 2. Why is performance monitoring important? Ans: 1) Software-based performance monitors must compete for system resources that would otherwise be allocated to programs that are being evaluated. This can result in inaccurate performance measurements. Hardware performance monitors operate in parallel with other system hardware, so Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 6
that measurement does not affect system performance. 2) Performance monitoring enables administrators to identify inefficiencies, using data describing how the system is functioning.
1. Why is it important to identify bottlenecks in a system? 2. Thrashing is due to the saturation of what resource? How would an operating system detect thrashing? Ans: 1) Identifying bottlenecks allows designers to focus on optimizing sections of a system that are degrading performance. 2) Main memory. The operating system would notice a high page recapture ratethe pages being paged out to make room for incoming pages would themselves quickly be paged back into main memory. A feedback loop is a technique in which information about the current state of the system can affect arriving requests. These requests can be rerouted if the feedback indicates that the system might have difficulty servicing them. Feedback can be negative, in which case arrival rates might decrease, or positive, in which case arrival rates might increase. 1. Explain how negative feedback could improve read performance in Level 1 (mirrored) RAID. 2. How does negative feedback contribute to system stability? Ans: 1) If one of the disks in a mirrored pair contains a large queue of read requests, some of these requests could be sent to the other disk in the pair if it contains a smaller queue. 2) Negative feedback prevents one resource from being overwhelmed while other identical resources lie idle. Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 7
1. In some large server systems, users communicate requests to a "front-end" server. This server accepts the user's request and sends it to a "back-end" server for processing. How could a front-end server balance request loads among a set of equivalent back-end servers using feedback loops? 2. This section describes how positive feedback can intensify thrashing. Suggest one possible solution to this problem. Ans: 1) Back-end servers with a long queue of requests might send negative feedback to the front-end sever, and back-end servers that are idle might send positive feedback. The front-end server can use these feedback loops to send incoming requests to underloaded instead of overloaded servers. 2) The system can monitor the number of page recaptures and refuse to admit further processes beyond a certain threshold. 1. What trade-offs must be weighed when including single instructions that perform complex routines in an instruction set? 2. Why is the choice of an ISA important? Ans: 1) The penalty for adding these instructions is more complex hardware, which might slow the execution of other more frequently used instructions; the benefit is faster execution of these complex routines. 2) The ISA specifies a programming interface between hardware and low-level software; therefore, it affects how easily code can be generated for the processor and how much memory that code occupies. Also, the ISA directly affects processor hardware, which influences cost and performance. 1. (T/F) The widespread use of high-level programming languages eliminated the usefulness of complex instructions. 2. What was the motivation behind the CISC processor-design philosophy? Ans: 1) False. Additional complex instructions were created to fit well with the needs of optimizing compilers. 2) Early operating systems were written primarily in assembly code, so complex instructions simplified programming, because each instruction performed several operations. Institute of Business and Technology Developed by:Adnan Alam Khan([email protected]) Department of Computer Science & Information Technology 8
1. Why do RISC processors exploit pipelines better than CISC processors? 2. Why does context switching require more overhead on RISC than on CISC processors? Ans: 1) RISC instructions are of fixed length and generally require one cycle to execute, so it is easy to overlap instructions such that most stages in the pipeline are performing meaningful work. The variable- length instructions of CISC processors have the effect of idling sections of the pipeline not needed for simple instructions. 2) RISC processors contain many more registers than CISC processors, requiring a greater number of memory transfers during a context switch. 1. Despite increased additional hardware complexity, what is the primary characteristic that distinguishes today's RISC processors from CISC processors? 2. (T/F) RISC processors are becoming more complex while CISC processors are becoming simpler. Ans: 1) Most of today's RISC processors continue to provide instructions of uniform length that require a single clock cycle to execute. 2) False. RISC processors are indeed becoming more complex. Although CISC processors incorporate RISC design philosophies, their hardware complexity continues to increase.