In particular, many optimization algorithms for linear optimization exhibit poor performance when applied to the Klee–Minty cube. In 1973 Klee and Minty showed that Dantzig's simplex algorithm was not a polynomial-time algorithm when applied to their cube.[1] Later, modifications of the Klee–Minty cube have shown poor behavior both for other basis-exchange pivoting algorithms and also for interior-point algorithms.[2]
Description
The Klee–Minty cube was originally specified with a parameterized system of linear inequalities, with the dimension as the parameter. The cube in two-dimensional space is a squashed square, and the "cube" in three-dimensional space is a squashed cube. Illustrations of the "cube" have appeared besides algebraic descriptions.[3] The Klee–Minty polytope is given by:[4]
This has variables, constraints other than the non-negativity constraints, and vertices, just as a -dimensional hypercube does. If the objective function to be maximized is
and if the initial vertex for the simplex algorithm is the origin, then the algorithm as formulated by Dantzig visits all vertices, finally reaching the optimal vertex .
Computational complexity
The Klee–Minty cube has been used to analyze the performance of many algorithms, both in the worst case and on average. The time complexity of an algorithm counts the number of arithmetic operations sufficient for the algorithm to solve the problem. For example, Gaussian elimination requires the order of operations, and so it is said to have polynomial time-complexity because its complexity is bounded by a cubic polynomial. There are examples of algorithms that do not have polynomial-time complexity. For example, a generalization of Gaussian elimination called Buchberger's algorithm has for its complexity an exponential function of the problem data (the degree of the polynomials and the number of variables of the multivariate polynomials). Because exponential functions eventually grow much faster than polynomial functions, an exponential complexity implies that an algorithm has slow performance on large problems.
Modifications of the Klee–Minty construction showed similar exponential time complexity for other pivoting rules of simplex type, which maintain primal feasibility, such as Bland's rule.[6] Another modification showed that the criss-cross algorithm, which does not maintain primal feasibility, also visits all the corners of a modified Klee–Minty cube.[7] Like the simplex algorithm, the criss-cross algorithm visits all 8 corners of the three-dimensional cube in the worst case.
Path-following algorithms
Further modifications of the Klee–Minty cube have shown the poor performance of central-path–following algorithms for linear optimization, in that the central path comes arbitrarily close to each of the corners of a cube. This "vertex-stalking" performance is surprising because such path-following algorithms have polynomial-time complexity for linear optimization.[8]
Average case
The Klee–Minty cube has also inspired research on average-case complexity. When eligible pivots are made randomly (and not by the rule of steepest descent), Dantzig's simplex algorithm needs on averagequadratically many steps (on the order of.[9] Standard variants of the simplex algorithm take on average steps for a cube.[a] However according to Fukuda & Namiki (1994), when it is initialized at a random corner of the cube, the criss-cross algorithm visits only additional corners.[11] Both the simplex algorithm and the criss-cross algorithm visit exactly 3 additional corners of the three-dimensional cube on average.
^More generally, for the simplex algorithm, the expected number of steps is proportional to for linear-programming problems that are randomly drawn from the Euclideanunit sphere, as proved by Borgwardt and by Smale.[10]
Borgwardt, Karl-Heinz (1987). The simplex method: A probabilistic analysis. Algorithms and Combinatorics (Study and Research Texts). Vol. 1. Berlin: Springer-Verlag. ISBN978-3-540-17096-9. MR0868467.
Klee, Victor; Minty, George J. (1972). "How good is the simplex algorithm?". In Shisha, Oved (ed.). Inequalities III (Proceedings of the Third Symposium on Inequalities held at the University of California, Los Angeles, Calif., September 1–9, 1969, dedicated to the memory of Theodore S. Motzkin). New York-London: Academic Press. pp. 159–175. MR0332165.
Roos, C. (1990). "An exponential example for Terlaky's pivoting rule for the criss-cross simplex method". Mathematical Programming. Series A. 46 (1): 79–84. doi:10.1007/BF01585729. MR1045573. S2CID33463483.