New iterative methods for dense linear systems

. Solving dense linear systems of equations is quite time consuming and requires an efficient parallel implementation on powerful supercomputers. Du, Zheng and Wang presented some new iterative methods for linear systems [Journal of Applied Analysis and Computation, 2011, 1(3): 351-360]. This paper shows that their methods are suitable for solving dense linear system of equations, compared with the classical Jacobi and Gauss-Seidel iterative methods.


Introduction
Linear system plays an important role of applications in engineering and scientific computing such as boundary element methods, quantum mechanical problems and large least squares problems [1,2,7,12,16,17]. Large sparse linear systems can be solved efficiently by iterative methods, especially those based on a Krylov subspace. However, for large dense linear systems, it is hard to develop good numerical methods. The linear systems usually have the following form of linear equations where A is nonsingular, x is unknown and b is known and nonzero. The case where A is dense can be solved numerically by direct methods, iterative methods and parallel methods [11]. Early in the study for solving (1), the main methods are direct methods [9,10]. Then the iterative methods become more popular [5,13]. Recent years, the parallel methods are more and more presented [4,6,8]. Many recent studies focus on exploiting the rank structures in the systems. The hierarchically semiseparable (HSS) representations are shown to be very useful for some dense problems such as Toeplitz matrices and certain discretized matrices (e.g., discretized integral equations and Schur complements in the factorizations of discretized PDEs). HSS matrices are closely related to other rank-structured representations such as sequentially semiseparable matrices and quasi-separable matrices [8]. For PCs (personal computers), the iterative methods are preferred.
Du, Zheng and Wang [3] suggested some new iterative methods for solving linear systems, and they showed that these methods, compared with the classical Jacobi and Gauss-Seidel methods, can be applied to more systems and have faster convergence. The new proposed methods are easy to construct and the convergence conditions are easy to check. They are convergent as long as the coefficient matrix is diagonally dominant, while the classical methods require that the matrix be either strictly diagonally dominant or irreducibly diagonally dominant. It was showed that the infinity norm of the iterative matrix of the new methods are less than or equal to that of the iterative matrix of the Jacobi method.
In this paper, the new methods presented by Du, Zheng and Wang [3] are discussed for solving dense linear systems. The theoretical results indicate that a dense system can be solved by the new methods when the coefficient matrix is diagonally dominant, while the classical methods require that the matrix is either strictly diagonally dominant or irreducibly diagonally dominant.
And numerical examples are provided to further show that they are suitable and efficient for dense cases.
The rest of the paper is organized as follows. In Section 2, the new methods presented in [3] are briefly introduced. In Section 3, the new methods are discussed for dense linear systems. Section 4 provides numerical examples to show the effectiveness and efficiency of the new methods. The conclusions are in Section 5.
The following theorem shows that the infinity norm of the iteration matrix of method (2) is less than or equal to that of the Jacobi method.
The convergence theorem for (2) is as below.
For Theorem 2.2, there is the following remark addressed.

Remark 2.3 [3].
(1) The size n of the matrix A has to satisfy n > 3.
(3) Notice that the iterative method (2) converges even if the matrix A is just diagonally dominant.
In contrast, there is the following convergence theorem for Jacobi and Gauss-Seidel methods, which can be found in [14,15].

Theorem 2.4. If A is strictly diagonally dominant, or irreducibly diagonally dominant, then both Jacobi and Gauss-Seidel methods converge regardless of the choice of the initial guess x (0) .
Du, Zheng and Wang [3] suggested another method as follows, where the shown entries are nonzero entries. (2) only converges for matrices with positive entries. Thus, the following theorem holds. Although Theorem 2.6 implies that the condition �� � � in Theorem 2.5 is not required, it is still true that the iterative method (3) is also suitable for dense linear system.

Theorem 3.2. For an n × n (� � �) dense linear system (1), if the coefficient matrix A is diagonally dominant and all entries are positive, then it can be numerically solved by the method (3).
In the next section, some numerical examples are provided to further illustrate that the methods (2) and (3) are suitable for solving dense linear system of equations (1) with diagonally dominant coefficient matrix. And the numerical results show that the new methods outperform the classical Jacobi and Gauss-Seidel iterative methods even though the coefficient matrix is strictly diagonally dominant.

Numerical experiments
In this section, three examples are given to illustrate the new methods (2) and (3) for solving dense linear system of equations (1). All performances run in MATLAB 7.12 with 2.93GHz CPU, 4.00GB RAM and 32-bit Windows 10 Professional. The initial guess is 0, and the stopping criterion is where ��� is the residual vector after k iterations. The numerical results are listed in Tables 1-3, where Jacobi, GS, I and II stand for Jacobi method, Gauss-Seidel method, method (2) and method (3)   In this example, the new iterative method (2) has less iterations and CPU time than Jacobi method, and has less CPU time than Gauss-Seidel method does; the method (3) is as good as Jacobi method and better than Gauss-Seidel method, as seen from Table 2  In Table 3, when it reaches the maximum iteration number 5000, Jacobi method does still not reach to the given precision and needs more CPU time. And the two new methods have same iterations to Gauss-Seidel method and less CPU time than Gauss-Seidel method.

Conclusion
In this paper, two new iterative methods are discussed for solving dense linear system, which is easy to establish and meet the convergence conditions. The theoretical results indicate that a dense system can be solved when the coefficient matrix is diagonally dominant, while the classical methods require either strictly diagonally dominant or irreducibly diagonally dominant. Numerical experiments show that the new methods are better than Jacobi and Gauss-Seidel methods.