1804

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

Automated Design Debugging with Maximum Satisability

Abstract As contemporary very large scale integration designs grow in complexity, design debugging has rapidly established itself as one of the largest bottlenecks in the design cycle today. Automated debug solutions such as those based on Boolean satisability (SAT) enable engineers to reduce the debug effort by localizing possible error sources in the design. Unfortunately, adaptation of these techniques to industrial designs is still limited by the performance and capacity of the underlying engines. This paper presents a novel formulation of the debugging problem using MaxSAT to improve the performance and applicability of automated debuggers. Our technique not only identies errors in the design but also indicates when the bug is excited in the error trace. MaxSAT allows for a simpler formulation of the debugging problem, reducing the problem size by 80% compared to a conventional SAT-based technique. Empirical results demonstrate the effectiveness of the proposed formulation as run-time improvements of 4.5 are observed on average. This paper introduces two performance improvements to further reduce the time required to nd all error sources within the design by an order of magnitude. Index TermsDebugging, diagnosis, satisability, verication, very large scale integration.

I. Introduction HE COMPLEXITY of digital designs in the semiconductor industry has increased exponentially over the last few decades. Even though innovations in design tools and methodologies have made signicant progress in reducing human error, designing a circuit free of functional errors remains a challenge. Functional verication and debugging tasks have established themselves as major bottlenecks in the design process, requiring up to 70% of the total design effort [1]. Due to the risk inherent in todays highly complex design ows, engineers are increasingly adopting automated design and verication tools to ensure correctness [2], [3]. However, once verication fails, localizing and rectifying the erroneous behavior (debugging) remains a predominantly manual task
Manuscript received September 27, 2009; revised March 30, 2010; accepted May 14, 2010. Date of current version October 20, 2010. This paper was recommended by Associate Editor R. F. Damiano. Y. Chen and S. Safarpour are with Vennsa Technologies, Inc., Toronto, ON M5V 3B1, Canada (e-mail: [email protected]; [email protected]). J. Marques-Silva is with the School of Electronics and Computer Science, University College Dublin, Beleld, Dublin 4, Ireland (e-mail: [email protected]). A. Veneris is with the Department of Computer Science and the Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON M5S 3G4, Canada (e-mail: [email protected]). Digital Object Identier 10.1109/TCAD.2010.2061270

lacking sufcient automation. As design complexity increases, larger design and verication teams and the usage of third party IP further complicate the debugging task. It is therefore not surprising that on average 60% of the verication effort is spent on debugging [4] and the costs of identifying the root cause of an error are growing rapidly. Automated debugging solutions are essential to accelerate these tasks. Techniques based on simulation [5], path tracing [6], and binary decision diagrams [7] have been proposed in the literature to enhance the efciency of error localization and diagnosis. More recently, the performance of engines based on Boolean satisability (SAT) [8] has encouraged further research into formal debugging techniques. The purpose of these techniques is to automatically identify the source of functional errors based on constraints specifying the circuits expected behavior. In the SAT-based debugging approach presented in [8], design errors are located by rst adding correction models to the circuit implementation. A correction model can be dedicated hardware that indicates whether a circuit component can be rectied to correct the circuit. The problem is then transformed into a Boolean formula in conjunctive normal form (CNF) and constrained using the expected behavior of the circuit. Solving the CNF problem using a Boolean SAT solver implicates a set of suspect error locations which could be responsible for the error. Based on the above formulation numerous derivative works such as [9], [10], and [11] have signicantly improved the performance and scalability of automated debuggers to make them applicable for industrial designs. However, enhancing the performance (run-time) and capacity (memory) of these techniques remain an ongoing challenge. Contemporary debuggers still have difculty handling large industrial designs and the correction models greatly increase the size of the CNF problem. Moreover, existing techniques are limited to identifying where the most likely error locations are in the design (spatial error locations). They do not determine when in the error trace the bug occurs (temporal error locations). Temporal information is very important for diagnosing and correcting the design [12] as they allow the designer to track down more easily when in the error trace the error occurs. In this paper, a novel approach to design debugging using partial maximum satisability (Partial MaxSAT) is presented [13], [14]. Similar to SAT problems, Partial MaxSAT problems are SAT problems expressed in CNF. However,

0278-0070/$26.00 c 2010 IEEE

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1805

instead of nding an assignment to a satisable CNF formula, MaxSAT solvers nd the largest satisable subset of CNF clauses in an unsatisable problem. In essence, the inverse of that subset identies a set of error locations which could be responsible for the bug. Our novel formulation allows us to identify all error locations both spatially and temporally, while signicantly reducing the size of CNF problems compared to SAT-based debugging. Our Partial MaxSAT formulation can improve the performance of debugging while providing solutions at a ner granularity. In addition, this paper also proposes two techniques to improve debugger performance. First, we present a method to quickly enumerate similar Partial MaxSAT solutions based on results obtained from the solver. These allow us to reduce the number of solver iterations required to nd all solutions. Next, we show how tuples of related clauses can be grouped such that they are treated as a single high-level constraint. These tuples increase the granularity of the solutions and reduce the search space of the problem, improving run time. Groupings based on modules and time frames allow for more relevant solutions to be found at lower error cardinalities. We demonstrate how groupings can be used to implement hierarchical debugging [15] with Partial MaxSAT to nd both module-level and gate-level error locations. Experimental results show that our approach can nd error locations 4.5 faster on average and using 80% smaller CNF problems. The optimization techniques presented allow us to identify all MaxSAT solutions using 87.5% fewer solver iterations while improving the run time per iteration by 1.6. As the performance of MaxSAT solvers is still improving at a rapid pace [16], our formulation will become more effective as engine technology progresses. The remainder of this paper is structured as follows. First, some background information on SAT-based automated debugging and MaxSAT solvers is provided. Then our MaxSAT formulation for combinational circuits is presented in Section III. The formulation is extended to nding spatial and temporal error locations in sequential circuits in Section IV. An algorithm to reduce the number of MaxSAT iterations to nd all solutions is presented in Section V. Section VI uses groupings to leverage the hierarchical structure of the circuit to improve performance. Finally the experiments, related works, and conclusion are provided in Sections VII, VIII, and IX, respectively. II. Background A. SAT-Based Automated Debugging Functional design debugging occurs at the stage of the design cycle when the register transfer level (RTL) implementation of the design has failed verication (simulation or formal). The output of the verication effort is an error trace proving the existence of a bug. In the context of this paper, design debugging seeks to locate all possible error sources in the implementation. In the SAT-based debugging formulation presented in [8], the circuit is rst enhanced with a correction model by adding a multiplexer at the output of each gate or module. The select

line of the multiplexer controls whether the correction model is active or inactive. If the correction model is inactive, the circuit behaves according to its implementation. If the correction model is active, the output of the gate is left unconstrained and can be replaced with a value that can correct the error. For sequential circuits, an iterative logic array (ILA) [6] is constructed by unrolling the circuit for the length of the error trace. The correction model is added as in the combinational case but multiplexer select lines for the same gate are shared across all time frames. The problem is converted into CNF and constrained using the stimulus vector and the expected output values from the error trace. Additional constraints (usually implemented using an adder) limit the number of correction models that can be simultaneously activated. This number is also known as the error cardinality Ng of the problem. Finding all satisfying assignment to the resulting formula effectively nds a set of functionally equivalent error locations, that if corrected could x the bug. Sets of error locations are said to be functionally equivalent if they cannot be functionally distinguished from each other under a given stimulus trace [6]. The debugger is limited to nding the set of all functionally equivalent error locations. Based on this formulation numerous advances to enhance the performance of debuggers have been proposed. The work from [9] presents a quantied Boolean formula (QBF) based debugging formulation using universal quantiers which allows for sequential circuit debugging without the need for an ILA. In [11], the concept of abstraction renement is used to reduce the size of the SAT problem and improve performance. In [10], the authors take advantage of unsatisable cores to speed up the debugging process for multiple fault diagnosis problems. This approach extracts a set of unsatisable cores from the CNF problem and prunes potential error locations not contained in any of the cores. A SAT-based exact debugger then nds the error locations from the reduced problem. B. Maximum Satisability This section reviews MaxSAT [16] and its extensions, and briey overviews recent algorithms for MaxSAT, capable of handling large complex problem instances. Given an unsatisable CNF formula , the MaxSAT problem consists of identifying an assignment to the problem variables such that the number of satised clauses from is maximized [17]. The MaxSAT problem is a well-known NP-Hard optimization problem. In the Partial MaxSAT [18] problem the CNF formula is organized into a set of hard clauses, which must be satised, and a set of soft clauses, which may or may not be satised, i.e., = H S . For Partial MaxSAT problems the objective is to nd an assignment that satises all the hard clauses and that maximizes the number of satised soft clauses. In the remainder of this paper, hard clauses will be represented in square brackets and soft clauses in round brackets. For example, consider the following formula: = [x1 + x2 ][x3 ] (x1 )(x2 )(x3 + x1 ).

(1)

1806

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

The rst two clauses are hard clauses, and so must be satised, whereas the remaining three clauses are soft clauses and may or may not be satised. In the recent past [17], the most effective MaxSAT algorithms have been based on branch-and-bound (B&B), supported by effective lower bounding and dedicated inference techniques. Nevertheless, most of the experimental evaluation associated with B&B MaxSAT solvers assumes random and handmade problem instances, which unfortunately often bears little relationship with hard industrial instances. Recent work has addressed alternative approaches, aiming the use of MaxSAT algorithms in industrial settings, and focusing on instances derived from realistic applications. The most effective algorithms are based on solving MaxSAT with unsatisable sub-formula identication and relaxation [18][20].

Fig. 1. Correct and erroneous combinational circuit. (a) Correct circuit. (b) Erroneous circuit.

gate at the output is mistakenly implemented using an OR gate. Given the input stimulus {i = 1, j = 1, k = 0} the actual output of the erroneous circuit is c = 1. The expected output response is c = 0. The MaxSAT debug formulation of this circuit expressed in CNF is given as follows: [i][j][k][c] A: (i + a)(j + a)(i + j + a)

III. Clause Level Debugging of Combinational Circuits In this section, the MaxSAT debug formulation for combinational circuits is presented. In order to express the debugging problem as a MaxSAT problem, the circuit must rst be converted into CNF. For the purpose of this paper we assume that circuits consist of single-output logic gates and CNF conversion occurs on a per gate basis in linear time [21], [22]. The goal of our debugging formulation is to identify a tuple of clauses in the CNF representation that are most likely responsible for the failure. A. Partial MaxSAT Formulation for Combinational Circuits The Partial MaxSAT formulation to debug a combinational circuit C given a correct specication is as follows: = [I][O] CNF (C) (2)

B: (k + b)(k + b) C: (a + c)(b + c)(a + b + c). For clarity, we included the gates represented by each set of clauses in the above CNF. Since the erroneous circuit cannot produce the expected response the problem is unsatisable. The maximum number of clauses that can be satised for this problem is 10 out of 12. A Partial MaxSAT solution for this problem consists of two UNSAT clauses. One of the solution sets that may be returned by a Partial MaxSAT solver is S1 = {C : (a + c), C : (b + c)}. For clarity, the gate to which each of the solutions maps to is given before each clause. Both clauses in S1 correctly imply that somehow correcting gate C will x the observed failure. We refer to this method of debugging as clause-level debugging. The number of clauses in the solution is known as the error clause cardinality of the solution. Clause-level debugging differs from SAT-based gate-level debugging techniques in that it seeks to nd a set of erroneous clauses instead of a set of erroneous gates. However, the motivation behind both of these techniques is identical. Just as gate-level debugging nds erroneous gates in order to map them to bugs in the RTL code, clause-level errors can also be mapped to gates in the netlist or code in the RTL. Clause-level debugging solves the debugging problem at a lower level of granularity since a single gate requires multiple clauses to specify its behavior. Intuitively this means that instead of the gate, rows in the gates truth table are identied as erroneous. A single erroneous gate may result in multiple UNSAT clauses in the MaxSAT solution. Consequently, while the cardinality of the clause-level solution is indeed minimal, the number of gates in the corresponding gate-level solution (its gate-level cardinality) might not be minimal. For instance, in Example 1, the solver could have alternatively returned one of three other possible solutions S2 = {A : (i + j + a), C : (b + c)} S3 = {C : (a + c), B : (k + b)} S4 = {A : (i + j + a), B : (k + b)}.

where C NF (C) is the CNF representation of the erroneous circuit, I represents the input constraints, and O are the corresponding expected output constraints. The input and expected output constraints are modeled using hard clauses (as indicated by the square brackets) as their values are assumed to be correct. Since I excites the erroneous behavior of C which is eventually observed at the output, the actual output of C does not match the expected output O. The formula is therefore inherently unsatisable. A MaxSAT solver nds an assignment to such that the set of satised clauses is maximized. The complement of this set represents the minimum set of clauses whose removal makes satisable. These clauses are the most likely error sources responsible for the unsatisability of the problem. For the remainder of this paper this complement set will be referred to as the MaxSAT solution. Each of these clauses in turn can be mapped to a set of gates that could be the cause of the erroneous behavior. Note that these clauses can originate from the same or different gates. Example 1: Consider the circuit given in Fig. 1. Fig. 1(a) represents the correct circuit implementation according to the specication. In the erroneous circuit of Fig. 1(b) the NOR

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1807

All solutions are of minimum error clause cardinality 2. However, the corresponding cardinality of gate-level solutions may vary. For S2 (gates A and C), S3 (gates B and C), and S4 (gates A and B) the gate-level cardinality is 2 whereas for S1 (only gate C) the gate-level cardinality is 1. The maximum number of UNSAT clauses due to a single gate is given by the following theorem. Theorem 1: Let G be a single-output logic gate that can drive both a value of 1 and 0 at its output. Then the maximum number of clauses that can be UNSAT in C NF (G) is mg 1, where mg is the number of clauses in the CNF representation of the gate. Proof: Let y be the output variable of gate G. In order to force y to a value of 1, C NF (G) must include a clause with literal y. Similarly, to force an output value of 0, one of the clauses in C NF (G) must include the literal y. Both the literals y and y must appear in C NF (G) at least once and they cannot appear in the same clause. Therefore, assigning any value to y causes at least one clause in C NF (G) to be satised. Thus, MaxSAT can return a maximum of (mg 1) clauses per gate. For instance, in Example 1, the behavior of gate A is represented by three clauses in the CNF. Thus mg = 3 for A. The maximum number of clauses given any variable assignment to the CNF for gate A is 2. B. Finding All MaxSAT Solutions In practice, has multiple solutions of minimum cardinality but the cardinality of the desired solution may not be minimal. Conventional SAT-based debuggers nd all solutions of a given error cardinality Ng to ensure completeness. Ng is dened as the maximum number of distinct gates contained in a solution. Similarly, clause-level debuggers need to nd all solutions up to a maximum cardinality Nc . As with Ng the error clause cardinality Nc is the maximum number of clauses in that are responsible for the bug. Existing MaxSAT solvers can iteratively provide these solutions but a mechanism to block previous solutions is needed to avoid duplicates. For solutions of cardinality one (i.e., the MaxSAT solution only contains one clause) converting the single clause to a hard clause effectively blocks the solution. For cardinality m > 1 however, converting each clause to a hard clause would prevent these clauses from occurring in other solutions. To illustrate this fact consider again Example 1. Suppose that the solution S2 is found in the rst MaxSAT iteration. Then converting the clauses (i+j+a) and (b+c) to hard clauses would not only block S2 but also the solutions S1 and S4 since each of these solutions contains one of those clauses. Relevant solutions might be inadvertently overlooked depending on the order in which solutions are found. Instead, the Partial MaxSAT problem must be reformulated such that at least one of the solution clauses evaluates to true. To prevent a specic set of clauses {Cl1 , Cl2 , . . . , Clm } from being returned as a MaxSAT solution, the following hard clause must be added to the CNF problem:
l l l Clb = [Cl1 + Cl2 + + Clm ].

l In (3), we let Cli denote the sets of literals in the respective Cli clause. Since the set of literals in Clb is the union of literals from all the solution clauses, at least one of the clauses in {Cl1 , Cl2 , . . . , Clm } must be satised. For instance, in Example 1 the solution S2 can be blocked by adding the clause [i + j + a + b + c] to . All solutions of cardinality Nc can be identied by continuously blocking solutions of minimum cardinality. Using Nc we can guarantee that all solutions of a certain gate-level cardinality Ng are found. The relationship between the solutions obtained when using either Nc or Ng for combinational circuits is given by the following theorem. Theorem 2: Let mcl be the largest value of mg for any gate in the circuit and let Eg be the set of all MaxSAT solutions of gate-level cardinality Ng . Let Ec be the set of all solutions of given a maximum error clause cardinality Nc = Ng (mcl 1). Then Eg Ec . Proof: By contradiction. Suppose that S Eg is a solution such that S Ec . Then |S| > Ng (mcl 1) where |S| denotes / the number of clauses in S. However, the maximum number of clauses that can be UNSAT for a single gate is mg 1 and mcl mg . Since Sc is of gate-level cardinality Ng , |S| Ng (mcl 1). Therefore Sg Ec contradicting our initial assumption. For Example 1, the highest value of mg is 3 so mcl = 3. To nd all gate-level solutions of cardinality 1, Nc must be 2. Note that the gate-level cardinality of the solutions can vary between Nc and Ng .

IV. Clause Level Debugging of Sequential Circuits A. Partial MaxSAT Formulation for Sequential Circuits In this section, the MaxSAT formulation for combinational circuits is extended to sequential circuits. Our formulation takes as its inputs the sequential circuit C, a sequence of stimulus vectors I1 , I2 , . . . , Ik and expected output sequence O1 , O2 , . . . , Ok . The variable k is the number of clock cycles in the error trace over which the behavior of C is modeled. The problem is also constrained by an initial state vector IS. The ILA of the circuit, otherwise known as the time frame expansion model, is constructed by unrolling the combinational portion of the circuit k times. This effectively translates the sequential problem into a combinational one. The ILA can then be converted into a SAT instance in CNF. The input to the MaxSAT solver is given by
k

=
i=1

[Ii ][Oi ] [IS] CNF (ILAk (C))

(4)

(3)

where I LAk (C) denotes the time frame expansion of C for k time frames. Example 2: Consider the erroneous circuit in Fig. 2(a). The correct circuit is derived by replacing gate A with an OR gate. The ILA representation of the circuit for three clock cycles is given in Fig. 2(b). For clarity, some irrelevant gates and constraints are omitted in the gure.

1808

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

roneous gates for a given error cardinality. Errors from the same or different gate-level sources are not distinguished at the clause level. For instance, in Example 2 solution S1 implicates two excitations of gate A as the cause of the problem. Solution S2 indicates that an excitation of gate A followed by an excitation of gate B could be the bug. In both cases, the number of excitations is two but no distinction is made whether these excitations come from the same or different gates. As a result, once an error clause cardinality Nc is specied, all clause-level errors are found irrespective of their corresponding gates. Nc can be more appropriately specied as Nc = Nep (mcl 1) (5)

Fig. 2. Erroneous circuit and its ILA representation. (a) Erroneous circuit. (b) ILA representation.

The initial state of the circuit is given by a0 = 0 and b0 = 0. The input vectors [i1 = 0, j1 = 1] and [i2 = 0, j2 = 1] cause the value at the fanout of gate A to differ from the correct values in time frames 1 and 2. The effects of these two error excitations are then propagated and observed in time frame 3 where the actual output (out3 = 1) of the trace differs from the expected output (out3 = 0). The MaxSAT solutions for Nc = 2 are as follows: S1 = {A1 : (i1 + a1 ), A2 : (i2 + a2 )} S2 = {A1 : (i1 + a1 ), B3 : (a2 + b3 )} S3 = {A1 : (i1 + a1 ), C3 : (b3 + c3 )} S4 = {B2 : (a1 + b2 ), A2 : (i2 + a2 )} S5 = {B2 : (a1 + b2 ), B3 : (a2 + b3 )} S6 = {B2 : (a1 + b2 ), C3 : (b3 + c3 )} S7 = {C3 : (b2 + c3 ), A2 : (i2 + a2 )} S8 = {C3 : (b2 + c3 ), B3 : (a2 + b3 )} S9 = {C3 : (b2 + c3 ), C3 : (b3 + c3 )}. From these solutions, S1 correctly implicates gate A as the source of the bug. The remaining solutions result from the presence of two propagation paths from the error excitations to the output signal out3 . One propagation path passes through the gates A1 B2 C3 out3 while the other path goes through A2 B3 C3 out3 . In the worst case the error could be excited in every clock cycle. Since a gate is replicated k times in I LAk (C) and the maximum number of clauses per gate is mcl , an error clause cardinality of Nc = Ng (mcl 1) k would be required to nd all solutions of cardinality Ng . It is clear that the performance of the Partial v formulation depends on the number of error excitations where the erroneous behavior is propagated to an output. Previous work [23] shows that errors in simulation traces are excited in temporal proximity to the outputs at which the bug is observed. From our experiments we also nd that for the majority of cases the error is excited for only a few clock cycles (often only once or twice) before its effects can be rst seen at the outputs. This experimental observation obviously works in favor of our formulation as MaxSAT nds the least number of error excitations required to observe the bug. Moreover, unlike conventional gate-level debuggers, our MaxSAT formulation is not limited to nding a set of er-

where Nep is the maximum expected number of gate-level error excitations and propagations for a given stimulus trace. Note that Nep and Ng describe two different concepts. Ng describes an estimated number of spatial error locations that exist in the design irrespective of time frames or clock cycles. Nep , however, describes an estimate of the maximum number of times the error is active in a particular error trace. In other words, the value of Ng relates to the number of error locations of the bug in the RTL, while Nep relates to the observed erroneous behavior of a particular simulation trace. Similar to Ng , the user can provide an estimate for Nep based on trace length and the complexity of the problem [8]. B. Extracting Temporal Information An advantage of clause-level debugging is that both gatelevel error sources and temporal bug information is provided. Since the circuit is replicated once for every clock cycle, each clause in the MaxSAT solution of represents an error location both spatially and temporally. Thus, each solution clause automatically indicates when during the error trace the bug is active. For instance, in Example 2, the clause (i1 + a1 ) in S1 indicates that gate A in time frame 1 could be one of the causes of the bug. The solution S1 therefore states that gate A must be active in time frames 1 and 2 to cause the error. Another advantage of the temporal information provided is that it establishes a more specic relationship between each of the gates in the solution. In gate-level debugging, when considering higher cardinality solutions, we can only determine that the locations in the solution act together to cause the error. However, the order in which each of these error locations act is unknown. Since temporal information is included in our MaxSAT formulation, we can determine the sequence in which the individual gates are related. For instance, solution S2 in Example 2 indicates that an excitation of gate A in time frame 1 followed by an excitation of gate B in time frame 3 may be the cause of the error in the design. In contrast, traditional SAT-based debugging would only indicate that the gates A and B can act together to cause the bug. Another approach is to aggregate the error information from all MaxSAT solutions by creating a histogram showing the number of suspected error sources found for each time frame. The frequency of solutions per time frame can provide hints about when the errors are excited in the trace. The error is more likely to be propagated through circuit elements close to the actual error excitation so generally the time frames with

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1809

the highest frequency of solutions are the best candidates. This aggregate temporal information can help the designer analyze the error trace [14]. Example graphs for different circuits are provided in Section VII for the experiments. V. Reducing MaxSAT Iterations for All Solution MaxSAT A. Finding Additional UNSAT Clauses Since our formulation improves the granularity of the error model and nds time frame information, the search space for solutions is signicantly larger compared to SAT-based debugging. Every gate is replicated over all time frames and each gate requires multiple clauses to represent. The number of MaxSAT iterations required to nd all solutions of cardinality Nc can be considerable. The purpose of this section is to improve on the technique given in Section III-B by applying a local search algorithm to explore the solution space [24]. In detail, we introduce a set of heuristics to derive more solutions based on existing MaxSAT solutions that originate from previous iterations of the basic algorithm. Consider again Example 2. The solutions S1 to S9 are due to the two propagation paths from the erroneous gate to the output. The respective variable assignments of each of these solutions only differ by a few variables. For instance, solution S1 only differs from S2 by the assignment to variable a2 . However, in order to nd all of these solutions a total of nine MaxSAT iterations is required. For larger circuits, each of these iterations can be computationally intensive. Consider an UNSAT clause clu for a given MaxSAT assignment. Since clu is UNSAT each of its literals evaluates to 0. Inverting the variable assignment for one of its literals would cause clu to become satised. However another set of with |clsi | 1 would become UNSAT. If clauses clsi this set only contains a single clause clsi = {cls } then the new assignment is another MaxSAT assignment for . The number of UNSAT clauses in the new assignment remains the same as in the original assignment and is therefore minimal. That is clu can be replaced with cls in the MaxSAT solution since the solution cardinality remains unchanged. We say that cls is a substitute solution clause for clu . An informal description of the technique applied multiple times to Example 2 is given below. Suppose the rst MaxSAT solution obtained is S1 . The clauses (i1 + a1 ) and (i2 + a2 ) are UNSAT. The variables a1 and a2 are therefore both set to 1 (i1 and i2 are constrained by hard clauses). The clause (i1 + a1 ) can be satised by setting a1 = 0. As a result exactly one other clause, (a1 + b2 ), becomes UNSAT. The clause (a1 + b2 ) is therefore a substitute solution clause for (i1 + a1 ) in S1 . Using this reasoning, the solution S4 can be directly derived from S1 . Continuing from this new MaxSAT assignment it is then possible to nd another substitute solution clause, (b2 +c3 ), by setting b2 = 1. Similarly the clauses (a2 +b3 ) and (b3 + c3 ) can be found as substitute solution clauses for (i2 +a2 ). These two sets of substitute clauses effectively identify S2 , S3 , S4 , and S7 as additional solutions. The algorithm to perform this local search is described in Algorithm 1. It takes an UNSAT solution clause clu , the

Algorithm 1: The nd substitutes algorithm Data: The CNF problem , an UNSAT clause clu , and variable assignment va Result: A set of substitute clauses and modications to va for each clause 1 find substitutes( , clu , va) 2 begin 3 stack.push([clu , ]) 4 subs [clu , ] while stack = do 5 6 [clt , mod] stack.pop() 7 invert assignment(va, mod) clsc = get connected clauses( , clt ) 8 foreach literal l clt do 9 n0 10 11 foreach clause cl clsc do 12 if sat literals(cl, va) = {l} then 13 nn+1 14 cls cl end 15 16 end 17 modl mod l if n = 1 and (subs [cls , modl ]) = ) then 18 19 subs subs [cls , modl ] stack.push([cls , modl ]) 20 end 21 end 22 23 invert assignment(va, mod) end 24 remove hard clauses(subs) 25 26 return subs 27 end

MaxSAT variable assignment va and the CNF problem as its inputs. The output of this algorithm is a set of substitute clause and assignment modication pairs. An assignment modication mod consists of a subset of variables from . Applying a modication mod to a variable assignment means that all the variables in mod are inverted to obtain a new assignment. In this case, the assignment modications can be applied to va to obtain new MaxSAT assignments. A stack is used by the algorithm to keep track of pairs that have not yet been examined by the algorithm. Whenever a new UNSAT clause clt is removed from the stack, a call to invert assignments on line 7 modies va such that clt is UNSAT under va. These modications are reverted on line 23. The search for solution clauses connected to clt is performed within the loop on line 9. The get connected clauses function nds all clauses in which have variables in common with clt . For each literal l clt and every connected clause cl clsc , the function sat literals nds all satised literals in cl given va. If l is the only satised literal in cl, then inverting the variable assignment of l would convert cl into an UNSAT clause. If only a single clause cls clsc becomes UNSAT, then cls is a substitute solution clause for clt .

1810

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

Fig. 3.

Deriving solutions from substitute clauses example.

The set of substituted clauses found by the algorithm is stored in subs. To avoid revisiting previously found solution clauses, line 18 checks cls against previously found solutions before adding the new solution to the stack. Finally, line 25 removes any hard clauses from the result. Note that the find substitutes function is limited to nding MaxSAT solutions which only differ by a single variable assignment from each other. Since we assume that our circuit consists of only single-output gates this method effectively nds clauses on an error propagation path consisting of functionally equivalent error locations. B. Deriving More Solutions Using Substitute Clause Sets Using the function find substitutes a new MaxSAT solution can be derived for each new substitute clause found. In this section, we examine how further solutions can be derived from combinations of substitute clauses. Consider for example a cardinality two MaxSAT solution: {clsa , clsb } and suppose that clta and cltb are substitute clauses for clsa and clsb , respectively. Then based on the find substitutes algorithm it is possible to obtain the solution {clta , clsb } by inverting a set of variables moda in the MaxSAT assignment. Similarly, {clsa , cltb } can be obtained by inverting a set of variables modb . From these observations we can deduce that inverting the set moda modb will convert {clsa , clsb } into satised clauses and {clta , cltb } into UNSAT clauses. Thus, {clta , cltb } is a MaxSAT solution unless there exists some other clause that becomes UNSAT due to the inversion of moda modb . For instance, the set of cardinality 2 solutions for Example 2 is essentially the cross product of the two substitute clause sets {(i1 + a1 ), (a1 + b2 ), (b2 + c3 )} {(i2 + a2 ), (a2 + b3 ), (b3 + c3 )}. The MaxSAT assignment for each solution is obtained by combining their respective assignment modications. Note that solutions obtained in this manner are not guaranteed to be valid MaxSAT solutions. Consider the following example. Example 3: The CNF of the circuit given in Fig. 3 is expressed as follows: [i][j][c][d][e] A: (i + a)(i + a) B: (j + b)(j + b) C: (a + c)(a + c) D: (a + d)(b + d)(a + b + d) E: (b + e)(b + e). Suppose that the initial solution is S1 = {A : (i + a), B : (j + b)}. Then nding the substitute clause (a + c) for (i + a)

by setting a = 1 will yield the solution S2 = {(a + c), (j + b)}. Similarly, setting b = 1 will yield S3 = {(i + a), (b + e)}. Setting both a = 1 and b = 1, however, increases the number of UNSAT clauses to three since (a + b + d) is now UNSAT. Example 3 illustrates that combinations of substitute clauses may not be valid solutions. However, in practice most combinations of clauses derived in this manner are MaxSAT solutions. To take advantage of this derivation method, we must check the validity of each derived solution. Since the number of clauses affected by the assignment modications is generally small, the run time required should be negligible. The optimized clause-level debugging algorithm to nd all MaxSAT solutions of cardinality Nc is given in Algorithm 2. After each iteration of the MaxSAT solver, find substitutes obtains a substitute clause set for each solution clause. The cross product of all substitute clause sets represents an overapproximation of alternative MaxSAT solutions. The function remove invalid solutions removes derived solutions whose assignment modications increase the number of UNSAT clauses. On line 12 the derived solutions stored in Et are then added to the solution set Es . The function block solutions blocks all solutions in Et as described in Section III-B. The algorithm terminates once a MaxSAT solution of cardinality higher than Nc is found. VI. Debugging Using Clause Groupings Clause-level debugging can be a very powerful technique for locating temporal and spatial error locations that are excited infrequently. However, depending on the type of bug, the error cardinality required to nd the relevant solution can be large. Increasing the error cardinality can reduce debugger performance since it is exponentially related to the complexity of the debugging problem [25]. If a large number of gates in the ILA representation are responsible for the error, a higher granularity error model might be required. This section presents a technique for increasing the granularity of our error model by grouping a set of clauses such that they are treated as a single high level constraint. The concept of clause groupings is introduced for the general MaxSAT case in [13]. Groupings can be created based on gates, modules and time frames. This allows us to increase the performance of our debugging algorithm at the cost of reducing the resolution of results. We can also use groupings to improve the performance of clause-level debugging by iteratively reducing the size of groups using hierarchical debugging from [15]. A. Creating Clause Groupings From the MaxSAT solvers point of view, the cost of setting all the clauses in the group to be UNSAT should be the same as the cost of allowing a single clause to be UNSAT. Groupings of clauses can be created by adding a single clause-selector literal to each clause in the group. To ensure that the problem remains UNSAT a unit clause consisting of the inverse of that literal is then added to . Consider, for instance, two clauses cl1 = (l1 +l2 +l3 ) and cl2 = (l4 + l5 ) where cl1 , cl2 . To create a grouping consisting of cl1 and cl2 the CNF is supplemented with an additional unit

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1811

Algorithm 2: The maxsat clause debug algorithm Data: Partial MaxSAT problem and maximum cardinality Nc Result: A set of suspect error sources Es 1 maxsat clause debug( , Nc ) 2 begin Es 3 4 [va, clss ] solve maxsat( ) while |clss | Nc and clss = do 5 6 Et 1 7 foreach clause cl in clss do 8 subs = find substitutes( , cl, va) 9 Et Et subs 10 end 11 remove invalid solutions( , va, Et ) E s = Es E t 12 block solutions( , Et ) 13 14 [va, clss ] solve maxsat( ) 15 end 16 return Es 17 end

Fig. 4. Different implementations of module level groupings. (a) Full module grouping. (b) Module input grouping. (c) Module output grouping.

our above example, the rst solution returned by the MaxSAT solver is (yc ). Using the find substitutes algorithm with clu = (yc ) would yield no substitute clauses since setting yc = 0 would unsatisfy more than one clause [(a + c) and (b + c)]. The benet of gate groupings in this case is that they allow the Partial MaxSAT solver to nd the error location at a lower cardinality. C. Groupings Based on Time Frame For sequential circuits, groupings can be created not only according to circuit structure but also across time frames. Thus instead of creating a new group for each time frame, groups can span across xed timing windows. The size of these timing windows represents a tradeoff between the accuracy of the temporal bug information and the complexity of the CNF problem. Ideally the window size should reect the expected number of consecutive bug excitations to reduce the cardinality of the solution. If no temporal bug information is desired, a single window may encompass the entire trace. D. Groupings Based on Modules A more sophisticated approach to groupings can be used for designs that are synthesized from RTL code. Hardware description languages (HDLs) such as Verilog and VHDL naturally group related design elements into modules or functions. Instead of identifying errors at the gate-level, module level groupings can be used to identify erroneous modules. These modules do not necessarily have to be explicit user dened modules but could also arise due to a simple add or subtract statement, or an if-else block. A combinational design consisting of two modules A and B is shown in Fig. 4(a). There are multiple approaches to creating a grouping for a module. The most intuitive way is to include all the gates of a module in a group as shown in Fig. 4(a). We can derive an alternative encoding for module-level groupings if we consider that modules in an RTL design represent implications between the input and output signals of the module. Module level debugging operates under the premise that removing the inputoutput relationship imposed by the module would satisfy the problem. In order to remove this relationship from the circuit however, it is not necessary to remove every clause in the module. Removing only the clauses at either the inputs or the outputs of the module will achieve the same result. The remainder of the module then can be expressed as hard clauses to reduce the search space for the MaxSAT solver.

clause (y). In the new CNF the following clauses:

cl1 and cl2 are replaced with

cl1 = (l1 + l2 + l3 + y) cl2 = (l4 + l5 + y). Suppose is satisable if cl2 and cl1 are removed. Then is satisable if the clause (y) is removed since setting the y = 1 will allow both cl1 and cl2 to be satised. Using this method to group connected gates, we can isolate regions in the circuit which could contain the bug. Bugs involving multiple clauses can therefore be identied without increasing the maximum cardinality. This technique effectively allows us to trade off (i.e., improve) run time at the expense of resolution. The granularity of the error model can be modied by varying the number of gates in each of these groups. B. Groupings Based on Gates One application of groupings is to include clauses belonging to the same gate in a group. This effectively sets Nc to be the same as Nep . Gate-level groupings for the circuit in Example 1 can be created as follows: [i][j][k][c] A: (i + a + ya )(j + a + ya )(i + j + a + ya ) B: (k + b + yb )(k + b + yb ) C: (a + c + yc )(b + c + yc )(a + b + c + yc ) (ya )(yb )(yc ). The variables ya , yb , and yc are the clause-selector variables. Removing any of the unit soft clauses will satisfy all the clauses in its respective gate. One disadvantage of using gate-level groupings is that the find substitutes algorithm from Section V cannot detect additional solutions if the number of clauses in the group that would otherwise be UNSAT is larger than 1. For instance for

1812

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

Algorithm 3: The hierarchical debug algorithm Data: The circuit graph C, Partial MaxSAT problem , maximum clause-level cardinality Nc , maximum module-level cardinality Nm and maximum hierarchical depth L Result: A set of module-level and clause-level error locations in the circuit 1 hierarchical debug(C, , Nc , Nm , L) 2 begin module solns 3 4 clause solns 5 t top level module(C) 6 queue.enqueue(t) 7 while queue = do 8 q queue.dequeue() p nested modules(C, q) 9 10 if level(q) L and p = then 11 M module debug(C, , p, Nm ) 12 module soln module soln M foreach module m M do 13 14 queue.enqueue(m) end 15 16 else 17 c clause debug(C, , q, Nc ) 18 clause soln clause soln c 19 end 20 end 21 return [module solns, clause solns] 22 end

Fig. 5. Multiple pass debugging example. (a) First pass. (b) Second pass. (c) Final pass.

Fig. 4(b) and (c) illustrates groupings using input and output gates, respectively. Empirically, we observe that grouping the gates at the module outputs yields the best performance results. While Fig. 4 demonstrates module groupings for a combinational circuit, the technique can be easily extended to the sequential case. Time frame groupings can be used in conjunction with module groupings to improve efciency. E. Multiple Pass Debugging Generally modules in digital designs are structured hierarchically with sub-modules nested inside parent modules. The simplest method to debug hierarchical designs is to create a new group for each nested module. To further boost performance, module-based hierarchical debugging [15], which iteratively traverses the module hierarchy by depth, can be used. Once a module-level solution of sufcient depth has been identied, clause-level debug can further rene the solution by identifying the exact erroneous gates and time frames. The advantage of this hierarchical approach is two-fold. First, since the intermediate results of this algorithm are suspect error modules, they can be immediately used by the designer for debug. Second, the search space for clause-level debugging is considerably smaller if only a few modules are being considered. Effectively, module-level debugging can act as a preprocess to speed up debugging for large circuits [10], [13]. Example 4 demonstrates how hierarchi-

cal debugging works for a circuit simulated for three clock cycles. Example 4: Consider the circuit given in Fig. 5. For the rst iteration [Fig. 5(a)], two groups, one for each of the two top level modules, are created. The modules are grouped across all time frames, hence no temporal information is obtained from the solutions. Using Partial MaxSAT, group GA is then identied as the only solution at this level. Since we only wish to nd solutions in module for A in the next iteration, module B is specied using only hard clauses. Suppose now that module A consists of the two submodules C and D. The groups GC and GD are created as shown in Fig. 5(b). In this iteration module D is identied as erroneous. Consequently, the clauses in group GC are also converted into hard clauses. Note that the search space for each of these iterations consists of only two modules. If all modules were assigned a group on the rst iteration and assuming module B contains no nested modules, the search space would consist of four groupings on the rst iteration and three on the second. Fig. 5(c) shows the case where clause-level debugging is performed for the last iteration. Since only the clauses in module D are soft clauses, the solver will exclusively return solutions from that module. The last iteration also provides the temporal error locations through its solutions. The hierarchical debugging algorithm which nds all module-level and gate-level solutions to an erroneous circuit C is given in Algorithm 3. The algorithm performs a traversal of erroneous modules in the design hierarchy using a queue starting with the top level module as shown on line 5. During each iteration of the loop, the algorithm nds the nested modules p to a set of erroneous modules q. The function module debug on line 11 then performs module level debugging by creating output groupings for modules in p as described in

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1813

TABLE I Problem Size and Run Time Comparison Between SAT-Based Gate-Level Debugging, Clause-Level Debugging Using MaxSAT, and Gate-Level Debugging Using MaxSAT
Error Trace # Gates divider 1 6291 divider 2 6291 fpu 1 86 020 fpu 2 87 144 hpdmc 1 18 444 hpdmc 2 18 444 mem ctrl 1 55 174 mem ctrl 2 55 174 mips789 1 73 600 mips789 2 38 524 mrisc 1 22 452 pipeline 1 5843 pipeline 2 6318 rsdecoder 1 15 738 rsdecoder 2 15 732 spi 1 3427 spi 2 3357 sudoku 1 46 668 Circuit # Time Frames 40 40 40 19 28 58 40 40 32 158 42 181 69 50 100 14 143 61 SAT-Based Debugging # # Time Lits Clauses in Sec. 9454k 3734k 39.31 9641k 3807k 37.59 153191k 60804k 2182.57 6013k 2441k 20.24 4677k 1859k 24.19 7464k 2896k 45.26 92935k 37173k 358.45 96131k 38441k 422.53 131773k 53315k 606.4 8569k 3280k 47.04 40694k 16164k 361.79 24776k 9306k 122.96 14377k 5581k 70.25 31771k 12623k 632.56 49634k 19286k 509.02 3005k 1223k 19.76 13697k 5207k 228.08 102302k 40551k 435.44 # Lits 1830k 1869k 27424k 774k 794k 1724k 14657k 15206k 17342k 2293k 6973k 7039k 3203k 5458k 10913k 351k 4036k 18469k # Clauses 736k 751k 10313k 316k 319k 696k 5896k 6113k 6954k 957k 2755k 2779k 1302k 2250k 4500k 141k 1635k 7587k Average Clause-Level Debugging % Clause Time Reduction in Sec. 80.29% 11.32 80.27% 13.72 83.04% 462.12 87.05% 8.64 82.83% 3.39 75.97% 11.67 84.14% 26.95 84.10% 31.86 86.96% 96.01 70.82% 278.51 82.95% 15.57 70.14% 22.52 76.67% 11.35 82.17% 17.87 76.67% 1616.14 88.47% 2.95 68.60% 142.45 81.29% 40.96 79.92% Average Speedup 3.47 2.74 4.72 2.34 7.14 3.88 13.3 13.26 6.32 0.17 23.24 5.46 6.19 35.4 0.31 6.7 1.6 10.63 4.49 Nc 1 1 3 2 1 1 1 2 1 1 1 1 1 1 1 2 1 1 Gate-Level Groupings Time Speedup Ng in Sec. 11.87 3.31 1 13.96 2.69 1 250.78 8.7 1 6.43 3.15 1 4.64 5.21 1 9.6 4.71 1 48.39 7.41 1 36.66 11.53 1 120.07 5.05 1 49.36 0.95 1 25.06 14.44 1 63.71 1.93 1 14.08 4.99 1 26.76 23.64 1 912.37 0.56 1 4.1 4.82 1 489.97 0.47 1 52.93 8.23 1 Average 4.04

Section VI-D. Similar to clause-level debugging, the MaxSAT solver is then used to iteratively nd all module level solutions p given the module-level cardinality Nm . A module level solution can be blocked by simply removing the grouping and converting all the clauses in the group to hard clauses. Clause-level debugging (line 17) is performed instead of module-level debugging once the module depth exceeds a certain user dened value L or if the module does not contain any other sub-modules. The clause debug function is a modied version of the maxsat clause debug algorithm from Section V-B which also takes the set of modules q and the circuit graph C as its input. Only the gates in q are specied using soft clauses while the remainder of the circuit is expressed with hard clauses. Thus, only solutions involving clauses from q are returned by the solver. Since the number of gates in the modules of q are a fraction of Cs total number of gates, the run time per iteration is reduced. VII. Experiments In this section, we experimentally demonstrate the effectiveness of our debug techniques. The techniques described in this paper are implemented using C++ using the solver from [19] as the underlying MaxSAT solver. All experiments are run on a 2.20 GHz Intel Core2 Duo machine with 4 GB of memory. For the remainder of this section, unless otherwise stated, averages are calculated by taking the geometric mean of the results. In total, two educational circuits (pipeline, sudoku) and eight circuits obtained from OpenCores.org [26] (divider, fpu, hpdmc, mem ctrl, mips789, mrisc, rsdecoder, spi) are presented. A single Verilog bug is inserted into each circuit at the RTL level. These may include inverting the condition in an if-statement, changing the operator in an expression or modifying a state machine to transition to an erroneous state. In rsdecoder 1 for instance the increment of a counter is changed from 1 to 2. Each circuit is simulated using a testbench comparing the simulated output values with the expected output values of

the circuit. As soon as an inconsistency is detected between the expected and observed values of the circuit, the simulation is terminated and the trace is recorded. The circuit is then synthesized and converted into CNF using the method in [21]. Some basic dangling logic removal is performed to reduce the problem size. The CNF is constrained using input and expected output values from the simulation of the correct circuit model as described in Section IV-A. Table I compares the effectiveness of our clause-level (Section IV-A) and gate-level (Section VI) debugging technique against SAT-based debugging [8] with MiniSat2 [27]. The formulation of [8] most closely resembles our technique since results are returned in terms of gates. To allow for a closer comparison, gates are grouped across all time frames since results obtained in this manner are identical to the error locations obtained using [8] for a given Ng . Columns 13 give the instance of the buggy circuit, the number of gates in the design, and the number of time frames in the ILA. Columns 46 then provide the number of literals, clauses, and the run time to obtain a single solution from of the debugging algorithm of [8]. The results for our MaxSAT debugging algorithm are given in columns 712. The rst two columns show the number of literals and clauses in the MaxSAT formulation. The reduction in the number of clauses compared to [8] is given in the third column. The run time results for a single MaxSAT iteration are given in the next column and the speed up compared to [8] is provided in column 11. The error clause cardinality is given in column 12. Finally, the results for our MaxSAT formulation with gate-level groupings across all time frames are provided in columns 1315. Looking at instance fpu 1 the number of gates is 86 020 and the number of time frames in the error trace is 40. The problem formulation for gate-level SAT-based debugging consists of 153 191 thousand literals and 60 804 thousand gates. The time it takes to solve this problem with MiniSat is 2182.57 s. Expressing this debugging problem using our Partial MaxSAT problem we see a reduction of 83.04% in the number

1814

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

TABLE II Iterative Solving Results Design divider 1 fpu 1 hpdmc 1 mem ctrl 1 mips789 1 pipeline 1 rsdecoder 1 # Iter 10 3 8 8 33 21 2 # Soln 65 68 26 14 312 195 2 Total Time 111.22 1088.68 30.82 231.7 3049.83 825.64 49.68 Average Soln/Iter 6.5 22.67 3.25 1.75 9.45 9.29 1 8.02 Time/Iter 11.12 362.89 3.85 28.96 92.42 39.32 24.84 32.78

and one new unit clause is added for each gate since the group spans all time frames. Therefore, the number of literals is increased by the number of clauses and the number of gates in the design from clause-level formulation. The number of clauses is only increased by the number of gates in the design. A. All Solution Partial MaxSAT Sections III-B and V discuss how all solutions for a given cardinality can be obtained. Table II summarizes the results for seven of our sample instances. Columns 24 give the number of MaxSAT iterations, the number of solutions found and the total run time required to nd all solutions. The average of the number of solutions per iteration and the run time per iteration are given in column 5 and 6, respectively. Considering all the circuits in Table II, the number of solutions found per iteration is 8. The time required to run find substitutes algorithm and verify the cross-product of substitute sets is negligible. Since each solution would have required a separate MaxSAT iteration the total run time is signicantly reduced. Our method is effective since errors are often propagated to a single fanout gate. In other cases, the value of the output can be changed by manipulating a single input without affecting the remainder of the circuit. For bugs with lower error clause cardinality and where the error propagates to multiple gates in its proximity, the method can be less effective. In our best case, all 68 solutions of fpu 1 are found using only three MaxSAT iterations. However, one reason why the number of solutions in this instance is large compared to other instances is related to the cardinality of the MaxSAT solution. With an error clause cardinality of three, many of the solutions for fpu 1 are different combinations of the same gates. The actual number of distinct error locations for fpu 1 is 29. B. Visualizing Temporal Information As described in Section IV-B, providing temporal debug information is crucial in design debugging. The aggregate temporal information extracted by our technique for the circuits mips789 1, hpdmc 1, and mem ctrl 1 is illustrated in Fig. 6. The frequency each time frame is implicated by a solution clause is shown in the histograms in Fig. 6. The likelihood of an error being active during the time frame is indicated by the height of the bars. The scatter plots underneath the histograms plot error locations vs. the time frame for which they are found. The purpose of these is to illustrate which error locations are implicated multiple times for different time frames by the debugger. The y-axis lists all unique error locations found by the algorithm and the x-axis shows the time frames during which these locations can be excited to cause the error. For instance, the bug in hpdmc 1 is created by removing a signal assignment in the RTL of a state machine. The actual error excitation occurs in time frame 24 since the removed signal was not assigned the proper value. The graph shows three highly likely erroneous time frames (2325) and two less probable ones (20 to 21). For this case, a total of 20 distinct error locations are found. Many error locations in time frame 24 are also solutions in time frame 25.

of clauses. The time it takes to solve this problem with Partial MaxSAT is 462.12 s, 4.72 times faster than the SAT-based solution. The error clause cardinality is 3. Using gate-level groupings, the run time is further reduced in this case to 250.78 s, an 8.7 times speed up. Note also that the cardinality for this solution is reduced from 3 to 1 compared to clause-level. Recall that for SAT-based debugging each gate is enhanced with a correction model (in the form of a multiplexer) and constraints are included in the CNF to limit the cardinality of the solution. The size of these constraints is quadratic with the number of gates in the circuit. Thus, the number of clauses in our MaxSAT formulation is considerably smaller in comparison. For the instances evaluated the size reduction is 80% on average. It should be noted that the SAT-based debugging technique from [8] could be modied to also include time frame information. In the original formulation of [8] multiplexer select lines for the correction model are shared for the same gate across all time frames. As a result, the number of inputs to the constraints logic enforcing error cardinality is equal to the number of actual gates in the circuit. In order to obtain time frame information, however, each of these select lines needs to be a separate signal. This increases the number of inputs to the constraints circuitry to (number of gates) (number of time frames). We did not include any experiments for this modied SAT formulation because for most of our circuits the CNF problem could not be loaded into memory due to the increased size of the constraints logic. Comparing against the original formulation [8], MaxSAT consistently outperforms SAT-based debugging for short traces. For longer traces however, the run time improvements can show greater variability due to the increased difculty of the problem. For instance, for mips 789 and rsdecoder 2, whose trace lengths are 158 and 100 clock cycles, respectively, MaxSAT performs signicantly worse than SAT. In the case of spi 2, whose trace length is 143 cycles, gate-level debugging using MaxSAT is slower than SAT-based debugging even though clause-level debugging is still faster. On the other hand, pipeline 1 still shows a signicant improvement in run time a trace length despite a trace length of 181 clock cycles. The geometric mean of the speed up for all examples compared to SAT-based debugging [8] is 4.49 and 4.04 for clause-level and gate-level debugging, respectively. The number of clauses and literals for gate-level MaxSAT is not given in the table but can be easily calculated. From Section VI, each clause in the group will have one additional literal. Each gate in the design will belong to its own group

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1815

Fig. 6. Aggregate histograms and scatter plots for (a) hpdmc 1, (b) mem ctrl, and (c) mips789 1.

For mem ctrl 1 one of the select signals of a MUX that controlled the data path is erroneously inverted. The solutions indicate that the bug could be xed in time frame 34 when the error is excited before the data is propagated to the output. Since the data propagates through different gates in each time frame the solution gates are mostly unique as indicated in Fig. 6(b). As the error propagates through the datapath the number of possible error sources also decreases. These graphs can allow the engineer to focus on specic regions in the design during specic time frames to correct the problem. It is not always the case, however, that the height of the histogram provides the actual error excitation. The bug for mips789 1 is created by changing the default assignment of a signal to a wrong value. From the scatter plot of Fig. 6(c) we see that some error locations are implicated multiple times by the debugger until time frame 19. In this case, examining these suspected locations instead of time frame 19 would be more benecial. Nevertheless, the shape of the graph can provide valuable information to the engineer about the nature of the problem. C. Grouping Clauses Fig. 7 depicts the average run time given different group sizes for all the circuits in Table I. Group sizes are given in terms of the number of gates in the group and groups span across all time frames. All the gates in the group are connected but groupings are created randomly. Fig. 7 shows that the average run time generally decreases as the size of the group increases. The largest performance gain occurs when increasing the group size to 50 gates per group but the marginal gain is relatively small when increasing the size of groups further. D. Hierarchical Debugging Table III summarizes the results of our hierarchical debugging algorithm from Section VI-E. The table only shows run

Fig. 7.

Group size vs. run time.

times for problem instances where the actual error location is located within a user dened sub-module of the design since the outputs of those are clearly dened. The purpose of these results is to compare the performance of clause-level debug with and without a hierarchical search. We show the time to nd the actual erroneous module and time to nd all clause-level error locations within that module. The number of gates in the targeted module is given in column 2. The time required to nd the module using groupings as described in Section VI-D is given in column 3. In our best example (fpu 1) the search space is reduced to 8320 gates (10% of total) and the search time is reduced by 5.4. In all instances, the reduction in run time is sufcient to compensate for the additional time required to nd the module. Comparing the average time per iteration from Table III against Table II a further speed up of 1.56 is observed. VIII. Related Works Debugging techniques based on formal engines have garnered much attention from the research community since SATbased debugging methodologies were introduced [28]. In order

1816

IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 29, NO. 11, NOVEMBER 2010

TABLE III Hierarchical Debugging Results

divider 1 fpu 1 hpdmc 1 mem ctrl 1 mips789 1 pipeline 1 rsdecoder 1

for the debugging problem to be formulated at a higher level of abstraction than Boolean. As with SAT-based debugging, QBF and SMT formulations for debugging do not provide any temporal debug information. Since these techniques use inherently different formal engines, they present alternatives to the proposed technique. Note that all alternative techniques tend to compare directly against the original SAT-based technique to provide a common reference. IX. Conclusion This paper introduced techniques for debugging circuits using Partial MaxSAT. We introduced an iterative method which accurately identies all spatial and temporal error locations in a sequential design given an error trace. Compared to SATbased debugging, our formulation improves the granularity of spatial error locations by identifying the entries in the truth table of each gate as possibly erroneous. This paper also provided additional techniques to speed up the process of nding all MaxSAT solutions. We described a fast search algorithm that quickly identies multiple substitute clauses and alternative solutions from a given MaxSAT solution. A technique to group related clauses was presented to further improve performance using hierarchical debugging. Hierarchical debugging quickly reduces the search space which, in turn, also reduces the the run-time for our Partial MaxSAT solver. The effectiveness of this method was demonstrated experimentally. For clause-level debugging we observed a 4.5 improvement in run time and a 80% reduction in problem size compared to a conventional SAT-based approach. Our search algorithm can nd 8 additional MaxSAT solutions per iteration with little computational overhead. Hierarchical debugging further improves the performance of clause-level debug by focusing on specic modules in the design and reducing the search time per iteration by 1.56. References
[1] P. Rashinkar, P. Paterson, and L. Singh, System-on-a-Chip Verication: Methodology and Techniques. Boston, MA: Kluwer, 2000. [2] N. Amla, X. Du, A. Kuehlmann, R. P. Kurshan, and K. L. McMillan, An analysis of SAT-based model checking techniques in an industrial environment, in Correct Hardware Design and Verication Methods (CHARME). Berlin/Heidelberg, Germany: Springer, 2005, pp. 254268. [3] A. Mishchenko, S. Chatterjee, R. K. Brayton, and N. E n, Improvee ments to combinational equivalence checking, in Proc. ICCAD, 2006, pp. 836843. [4] H. Foster, Assertion-based verication: Industry myths to realities (invited tutorial), in Proc. Int. Conf. CAV, 2008, pp. 510. [5] M. Abramovici, P. R. Menon, and D. T. Miller, Critical path tracing: An alternative to fault simulation, in Proc. DAC, 1983, pp. 214220. [6] M. Abramovici, M. Breuer, and A. Friedman, Digital Systems Testing and Testable Design. Rockville, MD: Computer Science, 1990. [7] S.-Y. Huang, A fading algorithm for sequential fault diagnosis, in Proc. DFT VLSI Syst., 2004, pp. 139147. [8] A. Smith, A. Veneris, M. F. Ali, and A. Viglas, Fault diagnosis and logic debugging using Boolean satisability, IEEE Trans. Comput.Aided Design, vol. 24, no. 10, pp. 16061621, Oct. 2005. [9] H. Mangassarian, A. Veneris, S. , M. Benedetti, and D. Smith, A performance-driven QBF-based iterative logic array representation with applications to verication, debug and test, in Proc. ICCAD, 2007, pp. 240245. [10] A. S low, G. Fey, R. Bloem, and R. Drechsler, Using unsatisable u cores to debug multiple design errors. in Proc. GLSVLSI, 2008, pp. 7782.

to extend the scalability of automated debugging techniques, extensions and alternative formal debugging methods have been proposed. Many contributions to the debugging problem focus on reducing the size of the CNF problem and can also be applied to the MaxSAT formulation presented here. In [10] for instance, unsatisable cores are used to speed up the debugging process for multiple fault diagnosis problems. This approach extracts a set of unsatisable cores from the CNF problem and prunes potential error locations not contained in any of the cores. A SAT-based exact debugger is then used to nd the actual error locations from the reduced problem. Another powerful technique to improve scalability is abstraction renement [11]. The technique proposed in [11] simplies design components according the designs structure and a pre-determined abstraction level. The debugging problem is then solved using the simplied model. Using the solutions returned, a renement process selectively reintroduces abstracted components into the circuit and the process is repeated until no further renements are necessary. To deal with long error traces, trace compaction techniques [12], [29], [30] and interpolants [31] can be used to reduce the number of time frames in the ILA. Trace compaction techniques take as their input an error trace and attempt to generate an alternative sequence of stimulus events that expose the bug within fewer time frames. These shortened traces can then be used by a automated debugger to locate the actual error. In [31], the authors use interpolants to reduce the number of time frames in the ILA by replacing sequences of time frames with an over-approximation of their constraints. This allows for a partitioning of the problem into smaller sub-problems such that they can be solved more easily. The majority of these techniques can be applied to improve the scalability of the MaxSAT formulation presented in this paper. Other approaches rely on different solvers such as QBF [9], [15] and satisability modulo theories (SMT) [32], [33] to improve scalability. The module based debugging approach presented in Section VI-E is largely based on the work of [15], which combines QBF and hierarchical debugging. The QBF based debugging approach [9] offers the advantage of a much smaller problem formulation since only a single time frame is instantiated. More recently, debugging formulations using SMT solvers have been proposed [32]. These formulate the debugging problem at the word level and use SMT solvers instead of SAT solvers to locate errors in the design. SMT solvers allow

CHEN ET AL.: AUTOMATED DESIGN DEBUGGING WITH MAXIMUM SATISFIABILITY

1817

[11] S. Safarpour and A. Veneris, Abstraction and renement techniques in automated design debugging, in Proc. DATE, 2007, pp. 11821187. [12] K. Chang, V. Bertacco, and I. Markov, Simulation-based bug trace minimization with BMC-based renement, in Proc. ICCAD, 2005, pp. 10451051. [13] S. Safarpour, M. H. Lifton, H. Mangassarian, A. Veneris, and K. A. Sakallah, Improved design debugging using maximum satisability, in Proc. Int. Conf. FMCAD, 2007, pp. 1319. [14] Y. Chen, S. Safarpour, A. Veneris, and J. Marques-Silva, Spatial and temporal design debug using partial MaxSAT, in Proc. GLSVLSI, 2009, pp. 345350. [15] M. F. Ali, S. Safarpour, A. Veneris, M. Abadir, and R. Drechsler, Postverication debugging of hierarchical designs, in Proc. ICCAD, 2005, pp. 871876. [16] Max-SAT 2009 [Online]. Available: http://www.maxsat.udl.cat [17] A. Biere, M. J. H. Heule, H. van Maaren, and T. Walsh, Eds., Handbook of Satisability (Frontiers in Articial Intelligence and Applications, vol. 185). Amsterdam, The Netherlands: IOS Press, Feb. 2009. [18] Z. Fu and S. Malik, On solving the partial MAXSAT problem, in Proc. Int. Conf. Theory Applicat. SAT, 2006, pp. 252265. [19] J. Marques-Silva and V. M. Manquinho, Toward more effective unsatisability-based maximum satisability algorithms, in Proc. Int. Conf. Theory Applicat. SAT, May 2008, pp. 225230. [20] J. Marques-Silva and J. Planes, Algorithms for maximum satisability using unsatisable cores, in Proc. DATE, Mar. 2008, pp. 408413. [21] T. Larrabee, Test pattern generation using Boolean satisability, IEEE Trans. Comput.-Aided Design, vol. 11, no. 1, pp. 415, Jan. 1992. [22] P. Manolios and D. Vroon, Efcient circuit to CNF conversion, in Proc. Int. Conf. Theory Applicat. SAT, 2007, pp. 49. [23] S. Safarpour, A. Veneris, and F. Najm, Managing verication error traces with bounded model debugging, in Proc. ASPDAC, 2010, pp. 601606. [24] L. Kroc, A. Sabharwal, C. P. Gomes, and B. Selman, Integrating systematic and local search paradigms: A new strategy for MaxSAT, in Proc. IJCAI, 2009, pp. 544551. [25] A. Veneris and I. N. Hajj, Design error diagnosis and correction via test vector simulation, IEEE Trans. Comput.-Aided Design, vol. 18, no. 12, pp. 18031816, Dec. 1999. [26] OpenCores. (2009) [Online]. Available: http://www.opencores.org [27] N. Een and N. Sorensson, An extensible SAT-solver, in Proc. Int. Conf. Theory Applicat. SAT, 2003, pp. 333336. [28] A. Smith, A. Veneris, and A. Viglas, Design diagnosis using Boolean satisability, in Proc. ASPDAC, 2004, pp. 218223. [29] S. Safarpour, A. Veneris, and H. Mangassarian, Trace compaction using SAT-based reachability analysis, in Proc. ASPDAC, 2007, pp. 932937. [30] Y. Chen, S. Safarpour, and A. Veneris, Optimal trace compaction with property preservation, in Proc. MWSCAS, Aug. 2009, pp. 12071210. [31] B. Keng and A. G. Veneris, Scaling VLSI design debugging with interpolation, in Proc. Int. Conf. FMCAD, 2009, pp. 144151. [32] S. Mirzaeian, F. Zheng, and K.-T. T. Cheng, RTL error diagnosis using a word-level SAT-solver, in Proc. ITC, Oct. 2008, pp. 18. [33] A. S low, R. Wille, G. Fey, and R. Drechsler, Evaluation of cardiu nality constraints on SMT-based debugging, in Proc. ISMVL, 2009, pp. 298303. Yibin Chen (S09) received the B.A.Sc. degree in computer engineering from the University of Waterloo, Waterloo, ON, Canada, and the M.A.Sc. degree in computer engineering from the University of Toronto, Toronto, ON, Canada. He is currently a Design Engineer with Vennsa Technologies, Inc., Toronto, ON. His current research interests include design debugging, formal verication techniques and formal engines, including Boolean satisability, MaxSAT, and SMT solvers.

Sean Safarpour (S01M09) received the B.A.Sc. degree in computer engineering from the University of British Columbia, Vancouver, BC, Canada, and the M.A.Sc. and Ph.D. degrees in computer engineering from the University of Toronto, Toronto, ON, Canada. He is currently the Chief Technology Ofcer with Vennsa Technologies, Inc., Toronto, where he is in charge of research and development. He is the author of dozens of conference and journal publications and one book on automated debugging. His current research interests include design debugging, formal verication techniques, and formal engines such as Boolean satisability, QBF, and SMT solvers.

Joao Marques-Silva (M95SM03) received the Engineers and M.S. degrees from the Instituto Superior Tecnico, Technical University of Lisbon, Lisbon, Portugal, in 1988 and 1991, respectively, the Ph.D. degree in electrical engineering and computer science from the University of Michigan, Ann Arbor, in 1995, and the Habilitation degree in computer science from the Technical University of Lisbon in 2004. He is currently a Stokes Professor of Computer Science and Informatics with the School of Electronics and Computer Science, University College Dublin (UCD), Beleld, Dublin, Ireland. Before joining UCD, he was with the University of Southampton, Southampton, U.K., and with the Technical University of Lisbon. His current research interests include algorithms for constraint solving and optimization, and applications in formal methods, articial intelligence, operations research, and bioinformatics. Dr. Marques-Silva serves as an Associate Editor of Integration: The VLSI Journal. He is a member of the Association for Computing Machinery. He received the 2009 CAV Award for fundamental contributions to the development of high-performance Boolean satisability solvers.

Andreas Veneris (S96M99SM05) received the Diploma in computer engineering and informatics from the University of Patras, Patras, Greece, in 1991, the M.S. degree in computer science from the University of Southern California, Los Angeles, in 1992, and the Ph.D. degree in computer science from the University of Illinois at Urbana-Champaign, Urbana, in 1998. In 1998, he was a Visiting Faculty Member with the University of Illinois at Urbana-Champaign until 1999 when he joined the Department of Electrical and Computer Engineering and the Department of Computer Science, University of Toronto, Toronto, ON, Canada, where he is currently an Associate Professor. He is the author of one book and holds three patents. His current research interests include CAD for debugging, verication, synthesis and test of digital circuits/systems, and combinatorics. Dr. Veneris has received several teaching awards and a Best Paper Award. He is a member of the Association for Computing Machinery, American Association for the Advancement of Science, the Technical Chamber of Greece, Professional Engineers of Ontario, and the Planetary Society.