Conflict-Driven Clause Learning (CDCL) SAT solvers can automatically solve very large real-world problems. IntSat is a new technique extending CDCL to Integer Linear Programming (ILP). For some conflicts, IntSat generates no strong enough ILP constraint, and the backjump has to be done based on the conflicting set of bounds. The techniques given in this article precisely analyze how and when that set can be translated into the required new ILP constraint. Moreover, this can be done efficiently. This obviously strengthens learning and significantly improves the performance of IntSat (as confirmed by our experiments). We also briefly discuss extensions and other applications.
One of the most remarkable successes of AI technology is probably the combination of heuristics, learning and novel datastructures that allow Conflict-Driven Clause Learning (CDCL) propositional SAT solvers to solve, fully automatically, very large and hard real-world industrial and scientific problem instances. It is therefore not surprising that for many years researchers have tried to produce effective extensions of CDCL able to handle richer constraint languages. In fact, SAT can be seen as the particular case of Integer Linear Programming (ILP) where all variables are binary (0–1) and constraints have the form (usually written as clauses, i.e., sets (disjunctions) of literals). Of course arbitrary ILP constraints can be encoded into clauses, but handling them natively opens the door to far more compact and efficient reasoning, learning and optimization.
Indeed, an extensive amount of work exists on CDCL-like techniques for pseudo-Boolean solving (a.k.a. 0–1 ILP), which considers Boolean variables, arbitrary linear constraints of the form and optimizing an objective function where the coefficients , are integers (see [7] for all background and references).
The new IntSat technique for ILP [6] goes another step beyond, introducing a CDCL technique for full ILP (arbitrary integer variables, linear constraints and objective). As reported in [6], IntSat appears to be the first ILP method that is competitive with state-of-the-art commercial tools such as CPLEX and Gurobi (which solve ILP by combining LP relaxations, simplex, branch-and-cut and many other methods, see [4]). This is the case already for the first IntSat prototype implementations (single-core, unlike its competitors), performing especially well on problems having a more combinatorial (as opposed to numerical) nature.
From CDCL to IntSat
SAT
IntSat
Clause
Linear constraint
0–1 variable x
Integer variable x
Positive literal x
Lower bound
Negative literal
Upper bound
Propagation
Bound propagation
Resolution inference
Cut inference
As very roughly expressed in Table 1, IntSat extends CDCL by taking decisions on bounds (instead of literals), exhaustive bound propagation, and at each conflict it attempts to obtain by cuts a new ILP constraint, use it to backjump, and to learn it to preclude similar future conflicts (see [6] for all details).
IntSat was the motivation for the work reported in this article, but it can also be useful for other applications. In IntSat, conflict analysis may terminate with a new constraint that is not strong enough to justify a backjump. In those cases, the backjump is done based on the so-called conflicting set of bounds, thus ensuring termination and completeness of the procedure. However, the newly asserted bound is not justified by a constraint, which may weaken the learning procedure. According to our experiments, for many conflicts where no suitable cuts exist, the techniques given here allow IntSat to translate the conflicting set of bounds into the required new ILP constraint, thus improving its performance (see the short section on experiments, Section 4).
The applications of this article go beyond IntSat: indeed efficiently expressing logical constraints is of course ubiquitous in ILP (see, e.g., [3, Section 7.3]).
In what follows, X is a set of integer variables , upper and lower bounded by . Lower case letters (possibly subscripted) x, y, z denote variables of X, and a, b, c, k, l, u denote integers. An integer linear constraint is an expression of the form . We consider the following basic problem. Given a disjunction D of bounds:
characterize the cases in which a single ILP constraint C exists such that has the same solutions in as . We stress the fact that such a C is not only a valid constraint in the sense of disjunctive programming [1,2,5], that is, entailed by in . Given the bounds B, the constraint C is also required to entail the disjunction D. In other words, in and under B, the constraint C and the disjunction D should be equivalent.
By default we assume that variables in D are pairwise distinct. (We will discuss the case of more than one bound for the same variable in D after the proof of Lemma 2.2.) Without loss of generality we will assume that for every in D, and for every in D (note that otherwise all solutions of B are solutions of D, and solves the problem). Similarly, we will assume that there are no bounds in D that contradict B, that is, no with , nor with , since then D can be simplified by removing such a bound.
We say that an upper bound in D is improving by k if , and similarly a lower bound is improving by k if .
This short article is structured as follows. The main result Theorem 2.3 proves that C exists if at most one of the bounds in D is improving by more than 1; if so, a linear algorithm for computing C is given. Lemma 2.2 proves that if at least two bounds in D improve by more than 1, then not even a set S of ILP constraints exists such that has the same solutions in as . An example is given in Section 3 and in Section 4 we report some first results. Section 5 discusses the situation where D is a disjunction of constraints. Finally, in Section 6, we conclude and discuss related and future work.
Theory
Let the disjunction D be as in the previous section. The following easy translation replacing any by simplifies matters: let for every in D, and for every in D (note the sign). After this translation all bounds occurring in D are of the form . We prepare by some easy observations in linear algebra. See for example [8] for the basics in linear algebra and linear programming.
Let be the standard basis of that is, , with the only 1 on position i. Let be positive rationals. The hyperplane through has equation:
Let . In the sequel, the points are the extremal points in which D is satisfied; D is not satisfied in . This can easily be verified, since , with the only 0 on position i. The hyperplane through , which is parallel to the previous one, has equation
If and , the latter equation becomes
Letbe positive integers. Let B be the set of boundsand D the disjunctionLet P be the bounded polyhedron indefined by B and the constraintThen P contains an integer point not satisfying D if and only if at least two ofare >1.
By default we assume that all variables are distinct. Since P is downward closed in each variable , we have if there exists a point such that . In particular, if P contains an all positive integer point, then P contains . Now we argue as follows: iff iff at least two of the integers are ⩾2 (as the sum has n terms). □
Let B and D be as in Section1. If at least two bounds in D improve by more than 1, then there exists no set S of ILP constraints such thathas the same solutions inas.
We first give the argument in the case in which D consists of two upper bounds that improve by more than 1, that is, with such that , . Note that the points , , satisfy D, but the point does not. All four points satisfy B, and the fourth point is in the convex hull of the first three, see Fig. 1. Since any set S of ILP constraints defines a convex subset of , no such set S can contain , , but not . It follows that has different solutions in as . This argument can easily be adapted to the case where . If D contains other bounds as well, then either for or for , not satisfying the bound in D. The argument above can now be carried out with these values for the variables occurring in the other bounds. □
Four points in B.
We now discuss what happens if some variable occurs more than once in D. First, in one pass over the disjunction D, by only retaining for each variable its least (since D is a disjunction) constraining upper bound and lower bound, one gets an equivalent disjunction with at most two bounds per variable. If there are two of them, one is an upper bound and one is a lower bound . If , then the disjunction D is always true and we can take . Therefore we can assume , and a similar argument as for Lemma 2.2 applies. To see this, we first construct a point satisfying the bounds in B but not satisfying D. Put for each such that both and in D. Put for each such that only in D. Put for each such that only in D. Assign an arbitrary value between and to for each not occurring in D. Call this point P. Now pick one specific such that both and in D. Define the point () to be point P with the coordinate for changed into (c). The points P, , all satisfy B; , satisfy D but P in the middle of , does not satisfy D. Since any set S of ILP constraints defines a convex subset of , no such set S can contain , but not P. It follows that has different solutions in as .
We now come to the main positive result. Define the standard sum for a disjunction D by
The number of variables in will be denoted by .
Let B and D be as in Section1. Assume at most one bound in D is improving by more than 1. Thenhas the same solutions inas, where the constraint C is defined as follows. If D is, where no bound inis improving by more than 1, thenIf D is, where no bound inis improving by more than 1, thenIn both casesis the standard sum for.
Let conditions be as above. We first normalize the problem in order to match the format of Lemma 2.1. Define new variables: if in D and if in D. Normalize D with the new variables into for all or . Normalize upper bounds for the new variables: if in D and (note the sign) if in D. If is , where no bound in is improving by more than 1, then Lemma 2.1 gives the constraint . Here is the sum of terms for all such that occurs in D. Note that . Moreover, substituting back in yields . Finally, multiplying by and substituting back yields the desired constraint in the lemma. The other case is similar. □
An example
We illustrate Theorem 2.3 and its proof by an easy example. Consider the following disjunction of bounds
where the original bounds are
We want to find a constraint that is logically equivalent to D given B.
Normalize the problem:
new variables: , , ;
new bounds: , , ;
new disjunction: .
Get the constraint, as in the proof of Theorem 2.3:
Express the constraint with the original variables:
For example, if and (values within the bounds, but not satisfying D), the constraint enforces (satisfying D). Conversely, if D is satisfied by values in B, the constraint is also satisfied.
Experiments
The techniques described here increase the number of early backjump cuts (see [6]) and hence at each conflict where this happens, a strictly stronger constraint is learned. The constraints learned at other conflicts are not affected. Therefore, one can expect an improvement of the learning and hence of the performance of IntSat. Here we briefly report on some first experimental results on the extent of this improvement for the IntSat ILP solver.
We took the 49 xx.lp optimization problems of Section 5 of [6], and ran our IntSat solver on each one of them with ten different random seeds (to make results statistically more significant). Each random seed was ran with a 60 s timeout, each one without and with the techniques given here, a total of 2 × 490 experiments. Percentages of early backjump cuts (see [6]) typically go up from around 3% (without) to 15% (with). The average (among the 10 seeds) best solution cost found improved in six out of 49 problems and got worse in one, but more importantly, the number of problems (out of 49) for which an optimal solution was found and optimality was proved for all 10 random seeds goes up from 31 problems (without) to 40 (with).
When D is a disjunction of constraints
We now discuss the case where D is not simply a disjunction of bounds, but a disjunction of constraints of the form:
where the are sums of monomials over X. Then for each (distinct) such one can infer lower and upper bounds from the lower and upper bounds B on X. Hence, by introducing a new variable for each , one gets the corresponding bounds on , and one can replace D by the disjunction over Z:
If the resulting problem consisting of and over Z fulfils the conditions of Theorem 2.3, this theorem shows us how to efficiently obtain a linear constraint over Z such that has the same solutions in as . It is not hard to check that, by replacing back in each by its corresponding , one gets a constraint C over X such that has the same solutions in as : any solution of can be extended into a solution of (by taking ), which gives us a solution of , which in turn gives us a solution of , and the same steps can also be done reversely.
For example, let B be , and let D be . Then after substitution, we get and , which fulfils the conditions of Theorem 2.3 since is improving by 1. The resulting is , and translating back, we get that C is .
Conclusions and related work
Disjunctions (and other logical constraints) are usually expressed with the help of conjunctions of several constraints and new 0–1 variables. For example, the disjunction , in the presence of bounds , both improved by 2 in D, can be encoded by the following constraints ( is a new 0–1 variable):
However, such encodings do not give equivalence, but only equisatisfiability. For example, satisfies D but not . Note that our method works with a single new constraint and without the need of auxiliary variables, and both requirements are necessary for our application in IntSat.
We also remark that it is well known that in many ILP problems a large percentage of variables are 0–1. Since no bound over a 0–1 variable is improving by more than one, it is not surprising that our method applies quite frequently in practice.
Finally, we also remark that, for each that is upper bounded in D, our result only requires the existence of an upper bound in B, and not a lower bound. Similarly, a lower bound of in D requires a lower bound for in B, and no upper bound. This makes the method slightly more general.
Footnotes
Acknowledgements
Asín’s research was funded by CONICYT, Chile, under FONDECYT project 11121220. Nieuwenhuis’ and Bezem’s research was supported by the Spanish grants TIN2013-45732-C4-3-P (MINECO) and TIN2010-21062-C02-01 (MEC/MICINN).
References
1.
E.Balas, Disjunctive programming, Annals of Discrete Mathematics5 (1979), 3–51.
2.
E.Balaset al., Disjunctive programming: Cutting planes from logical conditions, in: Nonlinear Programming, Vol. 2, 1975, pp. 279–312.
R.Nieuwenhuis, The IntSat method for integer linear programming, in: Principles and Practice of Constraint Programming, 20th International Conference, CP 2014, LNCS, Vol. 8656, 2014, pp. 574–589.
7.
O.Roussel and V.M.Manquinho, Pseudo-Boolean and cardinality constraints, in: Handbook of Satisfiability, A.Biere, M.Heule, H.van Maaren and T.Walsh, eds, Frontiers in Artificial Intelligence and Applications, Vol. 185, IOS Press, 2009, pp. 695–733.
8.
A.Schrijver, Theory of Linear and Integer Programming, Wiley, 1998.