A nonlinear domain decomposition (DD) solver is considered with respect to improved energy efficiency. In this method, nonlinear problems are solved using Newton’s method on the subdomains in parallel and in asynchronous iterations. The method is compared to the more standard Newton-Krylov approach, where a linear domain decomposition solver is applied to the overall nonlinear problem after linearization using Newton’s method. It is found that in the nonlinear domain decomposition method, making use of the asynchronicity, some processor cores can be set to sleep to save energy and to allow better use of the power and thermal budget. Energy savings on average for each socket up to 77% (due to the RAPL hardware counters) are observed compared to the more traditional Newton-Krylov approach, which is synchronous by design, using up to 5120 Intel Broadwell (Xeon E5-2630v4) cores. The total time to solution is not affected. On the contrary, remaining cores of the same processor may be able to go to turbo mode, thus reducing the total time to solution slightly. Last, we consider the same strategy for the ASPIN (Additive Schwarz Preconditioned Inexact Newton) nonlinear domain decomposition method and observe a similar potential to save energy.
CaiXCKeyesDE (2002) Nonlinearly preconditioned inexact Newton algorithms. SIAM Journal on Scientific Computing24(1): 183–200.
5.
CaiXCKeyesDEMarcinkowskiL (2002a) Non-linear additive Schwarz preconditioners and application in computational fluid dynamics. International Journal for Numerical Methods in Fluids40(12): 1463–1470. LMS Workshop on Domain Decomposition Methods in Fluid Mechanics (London, 2001).
6.
CaiXCKeyesDEYoungDP (2002b) A nonlinear additive Schwarz preconditioned inexact Newton method for shocked duct flows. In: Domain Decomposition Methods in Science and Engineering (Lyon, 2000), Theory and Engineering Applications of Computational Methods, Barcelona: International Centre for Numerical Methods in Engineering (CIMNE), pp. 345–352.
7.
ChenJDongY (2012) Energy optimization of representative barrier algorithms. Journal of Central South University19(10): 2823–2831.
8.
DoleanVGanderMJKherijiW, et al. (2016) Nonlinear preconditioning: How to use a nonlinear Schwarz method to precondition Newton’s method. SIAM Journal on Scientific Computing38(6): A3357–A3380.
9.
FarhatCLesoinneMLeTallecP, et al. (2001) FETI-DP: a dual-primal unified FETI method—part I: a faster alternative to the two-level FETI method. International Journal for Numerical Methods in Engineering50: 1523–1544.
10.
FarhatCLesoinneMPiersonK (2000) A scalable dual-primal domain decomposition method. Numerical Linear Algebra with Applications7: 687–714.
11.
GroßC (2009) A unifying theory for nonlinear additively and multiplicatively preconditioned globalization strategies: convergence results and examples from the field of nonlinear elastostatics and elastodynamics. PhD Thesis, Rheinische Friedrich-Wilhelms-Universität Bonn, Germany.
12.
GroßCKrauseR (2011) On the globalization of ASPIN employing trust-region control strategies—convergence analysis and numerical examples. Technical Report 2011-03, Institute Computer Centre, Universita della Svizzera Italiana.
13.
HackenbergDSchöneRIlscheT, et al. (2015) An energy efficiency feature survey of the Intel Haswell processor. In: 2015 IEEE international parallel and distributed processing symposium workshop, Hyderabad, India, 25–29 May 2015, pp. 896–904.
14.
HagerGTreibigJHabichJ, et al. (2013) Exploring performance and power properties of modern multi-core chips via simple machine models. Concurrency and Computation: Practice and Experience28(2): 189–210.
15.
HsuCHFengWC (2005) A power-aware run-time system for high-performance computing. In: Proceedings of the 2005 ACM/IEEE conference on supercomputing, SC ’05. Washington, DC, USA: IEEE Computer Society. ISBN 1-59593-061-2. DOI: 10.1109/SC.2005.3.
16.
HuberMGmeinerBRüdeU, et al. (2016) Resilience for massively parallel multigrid solvers. SIAM Journal on Scientific Computing38(5): S217–S239.
17.
InadomiYPatkiTInoueK, et al. (2015) Analyzing and mitigating the impact of manufacturing variability in power-constrained supercomputing. In: Proceedings of the international conference for high performance computing, networking, storage and analysis, SC ’15. New York, NY, USA: ACM, pp. 78:1–78:12. ISBN 978-1-4503-3723-6. DOI: 10.1145/2807591.2807638.
18.
Intel Corp (2016) Intel Xeon processor E5-2600 v4 product family specification update. Version: December 2016.
19.
KappiahNFreehVWLowenthalDK (2005) Just in time dynamic voltage scaling: exploiting inter-node slack to save energy in MPI programs. In: Supercomputing, 2005. Proceedings of the ACM/IEEE SC 2005 conference, pp. 33–33. DOI: 10.1109/SC.2005.39.
20.
KerbysonDJVishnuABarkerKJ (2011) Energy templates: exploiting application information to save energy. In: Proceedings of 2011 IEEE international conference on cluster computing CLUSTER 2011. DOI: 10.1109/CLUSTER.2011.33.
21.
KlawonnALanserMRheinbachO (2014) Nonlinear FETI-DP and BDDC methods. SIAM Journal on Scientific Computing36(2): A737–A765.
22.
KlawonnALanserMRheinbachO (2015) Toward extremely scalable nonlinear domain decomposition methods for elliptic partial differential equations. SIAM Journal on Scientific Computing37(6): C667–C696.
23.
KlawonnALanserMRheinbachO (2016) A highly scalable implementation of inexact nonlinear FETI-DP without sparse direct solvers. In: Numerical mathematics and advanced applications—ENUMATH 2015, Lecture Notes in Computational Science and Engineering, vol. 112. Cham: Springer, pp. 255–264.
24.
KlawonnALanserMRheinbachO (2018) Nonlinear BDDC methods with approximate solvers. Electronic Transactions on Numerical Analysis49: 244–273.
25.
KlawonnALanserMRheinbachO, et al. (2017) Nonlinear FETI-DP and BDDC methods: a unified framework and parallel results. SIAM Journal on Scientific Computing39(6): C417–C451.
26.
KlawonnARheinbachO (2007) Robust FETI-DP methods for heterogeneous three dimensional elasticity problems. Computer Methods in Applied Mechanics and Engineering196(8): 1400–1414.
27.
KlawonnARheinbachO (2010) Highly scalable parallel domain decomposition methods with an application to biomechanics. ZAMM Journal of Applied Mathematics and Mechanics90(1): 5–32.
28.
KlawonnAWidlundOB (2006) Dual-primal FETI methods for linear elasticity. Communications on Pure and Applied Mathematics59: 1523–1572.
29.
KlawonnAWidlundOBDryjaM (2002) Dual-primal FETI methods for three-dimensional elliptic problems with heterogeneous coefficients. SIAM Journal on Numerical Analysis40(1): 159–179.
30.
LavrijsenWIancuCde JongW, et al. (2016) Exploiting variability for energy optimization of parallel programs. In: Proceedings of the 11th European conference on computer systems, EuroSys ’16. New York, NY, USA: ACM, pp. 9:1–9:16. ISBN 978-1-4503-4240-7.
31.
LiDde SupinskiBRSchulzM, et al. (2010) Hybrid MPI/OpenMP power-aware computing. In: 2010 IEEE international symposium on parallel distributed processing (IPDPS), pp. 1–12. DOI: 10.1109/IPDPS.2010.5470463.
32.
LimMYFreehVWLowenthalDK (2011) Adaptive, transparent CPU scaling algorithms leveraging inter-node MPI communication regions. Parallel Computing37(10): 667–683.
33.
LiuLKeyesDE (2015) Field-split preconditioned inexact newton algorithms. SIAM Journal on Scientific Computing37(3): A1388–A1409.
34.
LiuLKeyesDEKrauseR (2018) A note on adaptive nonlinear preconditioning techniques. SIAM Journal on Scientific Computing40(2): A1171–A1186.
35.
MarcinkowskiLCaiXC (2005) Parallel performance of some two-level ASPIN algorithms. In: Domain decomposition methods in science and engineering, Lecture Notes in Computational Science and Engineering, vol. 40. Berlin: Springer, pp. 639–646.
NegrelloCGosseletPReyC (2018) Nonlinearly preconditioned FETI solver for substructured formulations of nonlinear problems. Available at: https://hal.archives-ouvertes.fr/hal-01774589. Working paper or preprint (accessed May 2018).
38.
RountreeBAhnDHde SupinskiBR, et al. (2012) Beyond DVFS: a first look at performance under a hardware-enforced power bound. In: 2012 IEEE 26th international parallel and distributed processing symposium workshops and PhD forum, pp. 947–953. DOI: 10.1109/IPDPSW.2012.116.
39.
RountreeBLownenthalDKde SupinskiBR, et al. (2009) Adagio: making DVS practical for complex HPC applications. In: Proceedings of the 23rd international conference on supercomputing, ICS ’09. New York, NY, USA: ACM. ISBN 978-1-60558-498-0, pp. 460–469. DOI:10.1145/1542275.1542340.
40.
TreibigJHagerGWelleinG (2010) Likwid: a lightweight performance-oriented tool suite for x86 multicore environments. In: Proceedings of PSTI2010, the first international workshop on parallel software tools and tool infrastructures, San Diego, CA, 13–16 September 2010.
41.
VishnuASongSMarquezA, et al. (2013) Designing energy efficient communication runtime systems: a view from PGAS models. The Journal of Supercomputing63(3): 691–709.
42.
YangWYangC (2008) Exploiting energy saving opportunity of barrier operation in MPI programs. In: Proceedings of the 2008 second Asia international conference on modelling simulation (AMS), pp. 144–149. DOI: 10.1109/AMS.2008.80.
