Kaczmarz method for saddle point systems

The Kaczmarz method is presented for solving saddle point systems. The convergence is analyzed. Numerical examples, compared with classical Krylov subspace methods, SOR-like method (2001) and recent modified SOR-like method (2014), show that the Kaczmarz algorithm is efficient in convergence rate and CPU time.


Introduction
A block 2 × 2 linear system of the form where A∈R m×m and B∈R m×n , called a saddle point system, arises in a wide variety of technical and scientific applications such as constrained optimization, the finite element method to Stokes equations, fluid dynamics and weighted linear least squares problem [2]. In 2001, Golub, Wu and Yuan [6] proposed the SOR-like method for solving symmetric augmented linear system (1). Then, many improved numerical methods are suggested [1,10,13]. Benzi, Golub and Liesen [2] also gave a review on numerical solution of saddle point problems in 2005. Pan, Ng and Bai [9] developed a deteriorated positive-definite and skew-Hermitian splitting (DPSS) preconditioner for nonsymmetric saddle point system (1), based on which, a series of DPSS-type methods [3,4,12] are investigated. The Kaczmarz algorithm is a popular iterative projection method [11] as it is simple to implement and the convergence is superior to classical splitting iterative methods such as SOR-like method. Many authors studied Kaczmarz methods for solving linear systems [7,8,11]. In this paper, the Kaczmarz algorithm is presented for symmetric saddle point system (1) with numerical comparisons to classical Krylov subspace methods [5] and splitting iterative methods SOR-like [6] (2001) and modified SOR-like [12] (MSOR-like, 2014).
The rest of this paper is organized as follows. In Section 2, the Kaczmarz method is presented and the convergence is analyzed. In Section 3, numerical examples are provided to show the effectiveness and efficiency of the algorithm. In Section 4, concluding remarks are drawn. Throughout the paper, I denotes the identity matrix.

Kaczmarz method
For saddle point problem (1), where A ∈ R m×m is symmetric positive definite, and B∈R m×n is of full column rank, Kaczmarz method can be proposed as follows, KACZMARZ METHOD. Given initial vectors x 0 ∈R m and y 0 ∈R n , for k = 0, 1, 2, · · · , the following iterative scheme is taken, denote the i k th row and j k th row of a matrix.
The convergence result for method (2) is as the following theorem.
where λ min (·) and λ max (·) are the the smallest and largest nonzero eigenvalues of a matrix.

Numerical experiments
Two numerical examples are discussed using Kaczmarz algorithm (2) compared with the classical SOR-like method [6], recent MSOR-like method [12] and Krylov subspace methods with preconditioner for solving symmetric saddle system (1). All runs are performed in MATLAB 7.12 on an Intel Core CPU 2.80 GHz (8.00 GB RAM) Windows 7 system. The first example is for solving weighted least squares problem [10].
Example 3.1. The weighted linear least squares problem Find: x * so that: induces the following saddle point system where = −1 ( − ). With = tridiag(1,2,1) ∈ × and = , the Kaczmarz method is applied to problem (4). Take = T as in [6] and the optimal parameter for SOR-like and MSOR-like methods. The numerical results are listed in Table 1, where the norm of absolute residual vectors is defined as the initial vector is taken as and all runs terminate if ≤ 10 −7 . Table 1 shows that Kaczmarz method converges faster than MSOR-like and SOR-like methods with the optimal parameter , that is, Kaczmarz method needs much fewer iterations and much less CPU time and and has much higher precision than MSOR-like and SOR-like methods which have similar convergence.
Next example in [1] is for solving the Stokes problem. Example 3.2. Consider Stokes problem: find u and p such that where Ω = (0,1) × (0,1) ⊂ R 2 , ∂Ω is the boundary of Ω, ∆ is the componentwise Laplace operator, u is a vectorvalued function representing the velocity, and p is a scalar function representing the pressure. By discretizing (5) with the upwind scheme, the following saddle point system of linear equations is obtained: In the experiment, the right hand side of and is chosen such that the exact solution of (6) � u * p * � = �  With m = n = 2q 2 , the initial vector and all runs terminated if ERR≤ 10 −7 , the numerical results are listed in Table 2, where Q = B T B and = (the optimal) for SOR-like and MSOR-like.
From the numerical results in Tables 2 and 3, it can be seen that the Kaczmarz method is superior to MSOR-like and SOR-like methods with the optimal parameter ω opt and the classical Krylov subspace methods, which is also shown from Table 1 in Example 3.1. Obviously, the Kaczmarz method requires fewer iterations and less CPU time and has higher precision than MSOR-like and SORlike methods.

Conclusion
In this paper, the Kaczmarz method is suggested for solving symmetric saddle point system, which is a popular iterative projection method and does not need multiplications of matrix and vector (different to classical splitting iterative method), thus easy to implement and more efficient. Numerical examples show that the Kaczmarz algorithm is much faster than the classical Krylov subspace methods and splitting iterative methods SOR-like and recent MSOR-like and has higher precision for a special case where B = I. As for general cases, additional techniques are expected, which will be presented in further work.