Mosul Dam: Challenges and Innovative Solutions

Reservoir siting, permitting, construction and operation can, at times, face insurmountable challenges. Mosul Dam, a multipurpose dam located in Northern Iraq, is constructed on a dissolvable anhydrite, gypsum and limestone formation. To mitigate long-term impacts to the foundation, the dam designers opted for a structural solution. They incorporated into the design a grouting gallery for long-term maintenance grouting of the foundation cutoff grout curtain. In 2016, concerns with the risk posed by the deteriorating foundation resulted in the execution of the Mosul Dam emergency grouting and maintenance project. This paper illustrates how the international community proactively came together to prevent an incident of potential tragic proportions well before it could manifest itself. The emergency grouting work is an excellent example of Governments investing in preventing a disaster rather than investing in fixing a disaster. The paper presents the general approach adopted for the execution of the emergency project and discusses how the scope of work (SOW) evolved to address the myriad of requirements, including the training and transfer of information and equipment to the Iraqi Ministry of Water Resources that could not have been foreseen during the scoping of the work. Additionally, this paper illustrates the success that can be achieved when team members share a strong commitment to cooperation and a determination to succeed. Résumé. Le choix du site, l'obtention des permis, la construction et l'exploitation des réservoirs peuvent parfois se heurter à des difficultés insurmontables. Le barrage de Mossoul, un barrage polyvalent situé dans le nord de l'Irak, est construit sur une formation d'anhydrite, de gypse et de calcaire dissolvable. Pour atténuer les impacts à long terme sur les fondations, les concepteurs du barrage ont opté pour une solution structurelle. Ils ont intégré à la conception une galerie d'injection pour * Corresponding author: nagesh.malyala@aecom.com © The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/). E3S Web of Conferences 346, 03025 (2022) https://doi.org/10.1051/e3sconf/202234603025 Sharing Water: Multi-Purpose of Reservoirs and Innovations


Introduction
Mosul Dam, in Northern Iraq (See figure 1), is constructed on a dissolvable anhydrite, gypsum and limestone formation. As the gypsum dissolves, it can form interconnecting openings in the foundation that can compromise the stability of the dam. To mitigate longterm impacts to the foundation, the dam designers opted for a structural solution. They incorporated into the design, a grouting gallery, for long-term maintenance grouting of the foundation grout curtain (See figure 2). Maintenance grouting has been performed continuously since the end of construction; however, concerns with seepage through the foundation prompted a board of consultants to recommend limiting the reservoir pool elevation from El.335 to El.319. This non-structural measure has been in place since 2006. Military hostilities resulted in pausing maintenance grouting temporarily in 2014, and the high risk posed by the deteriorating foundation drove the Government of Iraq to secure funding loans for emergency grouting and repairs from the World Bank and the Italian Government. The United States (US) Government also allocated assistance funding for the project. In 2016, the Iraqi Government entered into a design-build contract with Trevi, an Italian grouting contractor, for the emergency work. At the same time, and due to a separate Government to Government Agreement between the US and Iraq, the United States Army Corp of Engineers (USACE) was appointed as the Engineer for the project. AECOM was subsequently contracted by the USACE to support the Task Force [1][2][3].
The emergency effort completed in 2019 had the primary objective of constructing a single grout line across the dam; and conducting repairs to the bottom outlet structures and electromechanical equipment. This paper presents the general approach adopted for the execution of the project and discusses how the scope of work evolved to address the myriad of requirements that could not have been foreseen during the scoping of the work. Structural and nonstructural solutions adopted to address the evolving requirements are listed for context. Additionally, this paper illustrates the success that can be achieved when team members share a strong commitment to cooperation and a determination to succeed. The Emergency Mosul Dam Project is an excellent example of how the international community proactively came together to prevent a potential major disaster. References are provided for additional information with respect to the technical details of the project.  Figure 3 illustrates the overall composition of the project team. It is noted that Versar was also retained by the USACE to provide staff for the Mosul Dam Task Force. The USACE retained AECOM for quality assurance (QA) and engineering support for the project. In addition, the Dam Safety Modification Study Independent External Peer Review (IEPR) team shared information related to risk reduction design alternatives through face-to-face meetings at key project milestones. AECOM mobilized a team of approximately 40 experienced professionals (mainly engineers and geologists), supported by a large team of engineers and subject matter experts in the US.  Under the design-build contract, the SOW under each of these tasks was very broad. The specific details of the work were left up to the contractor subject to negotiations during the execution of the work.

Background and challenges
The magnitude of the challenges presented by the project were not completely known or understood until the start of the work. Minor site issues were exacerbated by overarching limitations including the following: -The project was located in an area with active armed conflict which limited the movement of equipment, materials and personnel. -The control of surrounding project area changed in the middle of the project from the Peshmerga forces (Kurdistan Region of Iraq) to the Iraqi Army bringing new travel and border custom requirements. -All specialized equipment, parts and materials were prepared for shipment in Italy, and trucked overland to Turkey and then to Iraq (See Figure 4). It could take up to eight weeks for necessary equipment and parts to be delivered to the site -The supply of quality cement was a difficult issue throughout the project. Cement deliveries were susceptible to interruptions due to a myriad of issues. In all instances the Owner played a critical role in the resolution of the issues. -Replacement of the gallery infrastructure (which was largely original from dam construction) was performed concurrently with emergency grouting. This was challenging due to the narrow width of, and limited access to, the gallery. It was necessary for the Contractor to install temporary infrastructure in significant sections of the gallery to facilitate the replacement of the old infrastructure. -The contractor's labor force consisted of over 800 workers from at least six major nationalities and including several additional languages. The Contractor, AECOM and the Owner staff were Italian, American, and Iraqi, respectively. While all parties communicated in English, the team discovered cultural and linguistic differences complicated communication at times. -The emergency nature of the project limited the familiarization of the Contractor and AECOM with existing site conditions and historical site and grouting information prior to mobilization. This hindered initial planning efforts. The MoWR maintained a very significant library of historical data, drawings, reports and information; however, it took a period of time to become familiar with this information and to assimilate it. -Time zone differences between Italy, US and Iraq in addition to different workweek workdays between western and middle eastern calendars created difficulties in the timely coordination between site and reach back activities. -Camp construction took longer than expected. As such, there were times when housing was scarce which became a critical issue.
The above listed overall challenges, in addition to limited familiarity with site conditions, deteriorated gallery infrastructure, and unreliable power service to the site, could have derailed the execution of any similar project. The following factors attributed to the success of the Mosul Dam project: -All parties demonstrated a strong commitment to the success of the project -even where there could have been significant resistance. Even when contentious issues required resolution, everybody involved showed restraint and a diligent focus on resolving the problem. The Owner, while cautious, embraced new technologies, shared control of the site and was very supportive of all efforts throughout the project. The Contractor was not prepared for the level of scrutiny and fiduciary responsibility exerted by the Owner or the level of planning and scheduling imposed by the logistical limitations discussed above.

-
The fast pace of this emergency project brought additional stress to resolve issues quickly; however, with all parties present at the dam virtually every hour of the week, the team was able to make decisions relatively quickly, and when necessary, adjust if a better decision became apparent. This efficient decision-making process was invaluable as each day brought new issues to light that had to be discussed and resolved by the parties.
-Each party brought unique strengths to the project, that when combined, yielded a well-rounded execution team. The Contractor provided technical know-how of modern grouting techniques. The Owner had firsthand knowledge of the dam, its operation, site conditions and forty years of grouting experience grouting at the site.

-
The project had the endorsement of the Governments of Iraq, US and Italy and the support of the Coalition Forces. This support allowed the parties to focus on the successful completion of the project.

Background and challenges Emergency Grouting
Grouting started on the crest of the dam to provide time for initial removal of old infrastructure and installation of new infrastructure in the eastern portions of the Gallery. Within the gallery, grouting was rolled out in segments and it was performed continuously even during the time the grouting infrastructure was being replaced. While there were a myriad of logistic/construction issues at any given time, some of the more critical technical issues included [4-7]: -Grouting Method. Maintenance grouting at the dam had been historically performed following the nipple grouting technique -where a drilled grout hole is grouted by pumping grout from the top of the hole exclusively. The emergency grouting Contract called for the use of stage grouting. While the difference in the grouting methods were well understood by all parties, the specific execution steps and requirements of stage grouting were not as clear and proved to be a source of initial disagreements. The team overcame these issues through direct discussions and workshops attended by all parties. -Grouting Depth. Initially, determining grouting depths was an elusive issue due to diverging contract interpretations. However, since grouting was being done by redrilling previously grouted holes it soon became apparent to all parties that grouting to the previously grouted depths was the only logical approach. Thus, the depths for the downstream and upstream grouting line matched the depth of the historical maintenance grouting effort. Depths for additional vertical and inclined grout holes were determined based on geology (shallow dissolution layers), instrumentation information and observation of drilling results such as artesian flows and artesian pressures encountered during drilling. This resulted in a robust two-line grout curtain shored by the additional grouting of relatively shallow dissolution prone zones; -Grout Mixes. Stage grouting required approval of new grout mixes, grout characteristics, grouting methods and procedures and grouting pressures that were significantly different from the ones previously used with nipple grouting at the site. The Owner was initially cautious in granting its approval for the use of higher grout pressures, and the difference of opinions provided the opportunity for all parties to understand, debate and ultimately embrace the new methodologies established for maintenance grouting at the dam; -Grouting Control System. The Contractor developed "T-Grout", a proprietary computerized grouting control system to execute day-to-day grouting activities. It took the MoWR staff time to embrace the system and to be trained on its use. However, MoWR operators and managers have fully embraced the system and continue to utilize the system for day-to-day grouting activities; -Grouting Analysis. The project required the analysis and resolution of grouting issues daily (e.g., rod drops, collapses, artesian conditions, etc.). The Owner's experience and knowledge of the site was critical for the project team to understand and resolve each of these issues; -Equipment. The project required updates to purchased drilling and grouting equipment in addition to the procurement of specially designed equipment for the gallery and site conditions. The Owner provided significant operational experience of the site that was critical to optimizing the purchased equipment and to specifying new equipment specially designed for the gallery and site conditions; -Grout Delivery. The Contractor proposed and installed continuous grout delivery loop systems to deliver grout on demand from the mixing plants to the individual injectors (batching and grouting units) and back to the mixing plants. Some loops extended over 3 Km in total length. The Owner was initially skeptical of the performance of the loop systems and preferred on-call batch systems they had historically used at the site. A batch system delivers grout from the mixing plant to an agitator that is connected to a short local loop system that delivers grout to the injectors. The on-demand long loop systems proved very effective and efficient during emergency grouting due to the large number of grouting locations (as many as 20 batching/grouting units at one time) and high takes at the grout holes. Towards the end of the project the on demand long loop systems were reconfigured to an oncall local loop systems. The on-call systems proved to be more efficient for maintenance grouting operations (fewer number of grouting locations and generally smaller grout takes); -Grout Collar Replacement. The Owner resisted replacing protruding metal grouting collars (nipples) at each grout hole with new collars installed flush with the gallery floor because the protruding collars facilitated nipple grouting and they have proven to be able to contain the high hydrostatic pressures occurring at the grout holes. The Contractor considered the protruding collars a trip hazard, a cause for delays when moving equipment and a cause of damage to the equipment that need to track over them. The owner agreed to the change only after the collar design was modified to accommodate nipple grouting and the Contractor agreed to perform a pullout test on each of the new collars. All of the protruding collars (over 4,000 in total) were replaced with flush-mounted collars in the gallery.

Bottom outlet repairs
The bottom outlets consist of twin tubes located within the lower portions of the dam each capable of passing water at a rate of 1,200 cms. The system includes bulkhead gates at the upstream end well below the reservoir surface, two guard gates located roughly at the midpoint of the two tubes, two 10-meter diameter steel and concrete tunnels downstream of the guard gates, and twin regulating gates at the downstream end used to control flows from the reservoir. The SOW consisted of the inspection and refurbishment of the bulkhead gates, repair to the west guard gate, inspection of the bottom outlet (BO) tunnels, and refurbishment of two gantry cranes. The lifting, inspection and maintenance of the guard gates required the mobilization of a modular barge from Europe and its assembly at the site. The depth of the bulkhead gates below the reservoir level required the assistance of a diving team mobilized from Europe to lift the gates to the surface for inspection and maintenance. The inspection and maintenance of the bulkhead gates was completed without serious issues. Other features of the project required additional efforts as described below: -Inspection of the west bottom outlet tunnel guard gate revealed problems with the gate position indicator rod. Replacement of the rod and strengthening the rod indicator system required the procurement of specialty items from Europe. This work was expedited for the gate to be operational within one month from mobilization -an achievement which brought confidence to the whole team who had only been working together for a relatively short period of time at that point.
Other system upgrades were completed later; -Inspection of the west BO tunnel revealed seepage at the flexible joint between the tunnel and the guard gate structure. Repairs to the joint assembly was a complex endeavor due to the high hydrostatic pressure at the joint; -Gantry crane equipment and motors required extensive repairs and these repairs were concluded successfully. Partially in response to the successful repair of the initially selected cranes, the Owner requested refurbishing the gantry cranes at the spillway gates and additional work at the gantry crane outside the power plant; -Inspection of the bottom outlet dewatering pumps revealed the need for extensive repairs and upgrades to all electromechanical components of the dewatering system; -Inspection of the joints between concrete monoliths which make of a connector adit between the grouting gallery and gate tower structure, revealed significant expansion movement and the need to strengthen one joint; -Inspection of the intake to the water supply system to the gallery and grouting operations resulted in the procurement and installation of a new barge to support the intake pump as well as a new connecting pipeline; -Analysis, design and construction of new dentates installed at the point of discharge of the existing regulating gates on the bottom outlet to spread the energy of the plunging water jets and to prevent further erosion of the plunge pool; and -Repair to the guard gate tower elevator.

Instrumentation System
Mosul Dam is equipped with an extensive instrumentation system. Parts of the system were under construction during the emergency grouting work [8]. This presented multiple challenges as follows: -Piezometer installations conflicted at times with the grouting work due to space constraints in the gallery, conflict with the sequence of grouting, and with the availability of drilling rigs. Extensive coordination was required to resolve these issues; -Existing piezometers tended to be damaged or impacted by concurrent grouted activities. A roving team of drillers and technicians with special equipment was set up to routinely inspect and clean grouted piezometers; and -Protruding extensometers and piezometer gages and wires were prone to damage due to equipment movement in the narrow gallery. Replacement of some of these instruments was necessary. All extensometers and gages were protected with metal enclosures and cables and pipes/hoses were ultimately grouted in slots cut into the gallery concrete walls to avoid open exposure.

Non-structural solutions Precision Ground Surveys and Interferometric Synthetic Aperture Radar Surveys
US Government directly funded precision ground surveys and interferometric synthetic aperture radar surveys of the site. The objective of these tasks was to monitor dam performance during the emergency grouting effort. The SAR survey was executed without major issues and was very effective in identifying very small ground movements throughout the site. Data from the SAR surveys was used to guide the subsurface investigation on the downstream side of the dam [9]. The precision ground surveys required extensive work to reconcile survey control information from the time of construction (site specific coordinate system), and the control system utilized by MoWR with new Iraq Datums. The precision ground vertical movement surveys were very challenging due to the length of the underground gallery and presence of ongoing drilling and grouting activities in the gallery (gallery loops required traversing into, along, and out of the gallery -a distance of over 6 km). The work also required coordination with MoWR Baghdad based surveyors. AECOM mobilized state of the art survey equipment which was later transferred to MoWR and AECOM trained MoWR local surveyors on the use of the equipment.

Training and Integration of MoWR Worker
The SOW included the Contractor training up to 250 MoWR workers but the type of training and the curriculum of the courses was not clearly specified. Workers received classroom training and practical training by shadowing the Contractor. Liability and labor rule issues initially prevented the Contractor from fully integrating MoWR workers into the Contractor's crews; however, the Owner modified the Contract to include the integration of MoWR workers into the Contractor crews. Subsequently, MoWR workers started working for the Contractor (without labor issues or losing their status as employees of the Government of Iraq). Significant time was spent in selecting the employees to be trained, composing the crews and the gradual transition to 100% Iraqi crews and the assignment of these crews to work areas (e.g., gallery and crest) under the direct responsibility of MoWR engineers, foremen and drilling and grouting crews. A timeline for the transition of the execution of the work was developed for all critical activities at the site. This process occurred over a period of 18 months and was not easy to achieve as the Contractor retained full responsibility of the work (production and quality) until the conclusion of the contract.

Training and Integration of MoWR Technical Staff
Training was provided for mid-managers, engineers, geologists, and IT personnel on the areas of communication, IT, T-Grout, mixing plant management, maintenance shop, warehousing, testing laboratory, and QA activities and analysis (e.g., OPTV, WPT, flow meter, specifying grouting pressures, CADD, logging, etc.). The training was successful and required extensive coordination and management to document the attendance and progress of each of the staff to comply with Iraqi Civil Work laws and MoWR regulations. MoWR selected the staff for training based on qualifications. A steering group of representatives from the MoWR, USACE, AECOM and the Contractor oversaw the integration program. The steering group developed a culture of cooperation that extended through the ranks of the entire team such that everybody became a mentor of the integrated staff. It helped greatly that by the time the integration program was being implemented enough friendships had been developed among all team members that trust, a spirit of cooperation and a drive to complete this exercise for the future of the dam and MoWR success existed throughout the team.

Procurement of Parts and Consumables
The Owner modified the Contract to include the procurement of additional parts and consumables to sustain future drilling and grouting operations. A team of managers from MoWR, USACE, AECOM and the Contractor was established to develop lists of parts and consumables to be provided for all key site operations (e.g., maintenance shop, drilling and grouting, dewatering systems, etc.). The final lists were very detailed and clearly justified each item. The development process considered the logistics limitations for the procurement and delivery of the parts.

Warehousing
The development of the lists of parts and consumables highlighted the need to increase the capacity of the MoWR warehouse. MoWR identified additional buildings to house the parts and consumables to be procured as well as the need to order a kilometer length of shelving for the additional facilities. MoWR excelled at tracking equipment and parts at the site but relied on a traditional card/manual system. MoWR retained an independent local consultant to implement a new computerized system to track and manage its expanded inventory of equipment and parts. MoWR warehouse staff was incorporated into the Contractor warehouse operation for training in the use of computerized systems.

Refurbishing of MoWR Drilling Rigs
Early in the project, the need for additional drilling capacity for the timely completion of the emergency grouting work was reported from the field. It took several weeks for the Owner to agree to the transfer of the rigs; however, ultimately six rigs were transferred throughout the life of the contract for refurbishing and use on the drilling work. In some cases, the refurbishing effort consisted of the complete overhaul of the rig and in other cases, only minimal maintenance was required. However, due to logistical limitations associated with the procurement and delivery of parts; complete implementation of this initiative took over eight months to be completed. MoWR managers deserve significant credit for facilitating the transfer of the rigs to the Contractor as the additional rigs were instrumental in meeting the schedule considering the additional demand for equipment due to site investigations and instrumentation activities. Ultimately, at the end of the project, the Owner was provided additional drilling capabilities that they can use either at Mosul Dam or other dams within Iraq.

Transfer of T-Grout, GIS geological Model, QA applications and Project Digital Files
At the end of the project, the T-Grout software and all production files, the QA application software and technical documents and drawings generated during the project were transferred to the MoWR. Towards the end of the project thousands of files had to be screened to differentiate working files from final documents prior to transfer to the MoWR. The overall transfer of the T-Grout software and data base, QA application and data base, and engineering files was successfully completed with all systems being operational under MoWR oversight at the completion of the project.

Update of O&M Manuals
The O&M manuals of all refurbished electromechanical systems were updated with the new and refurbished equipment information and provided to the Owner.

Achievements
The project team successfully completed all contracted tasks. Additionally, the team completed new tasks that came to light during the execution of the project. Some of those tasks were unavoidable due to unexpected site conditions. Most of the additional tasks were executed to enhance the sustainability and reliability of the Mosul Dam operation for the midterm period following the completion of the emergency project. Those tasks could not have been executed without the collaboration and foresight of the entire team. Throughout the project, MoWR site managers cautiously approached the implementation of new technology and significant operational changes. This approach proved to be the correct one given the size of their operation, the makeup of their labor force, Iraqi labor laws and MoWR historical labor rules.
The collaborative efforts restored full operational capabilities of the dam, provided state of the art equipment, software and instrumentation, provided significant stockpiles of highquality supplies, materials and replacement parts and developed a highly skilled work force and engineering team. The ultimate success of this project was a result of overcoming significant technical, communication and institutional challenges combined with a significant effort to collaborate to overcome daunting security, logistics, personnel, integration, training, administration and financial challenges that changed, quite literally, on a daily basis.

Conclusions
The proactive actions supported by the US, Iraq and Italy at Mosul Dam are an excellent example of how to avoid costly potential catastrophes. Strong leadership by all parties is critical to the success of any project and even more so for projects with evolving requirements and of the scale and extent of Mosul Dam The Mosul Dam Project was staffed with experienced and knowledgeable personnel by all parties at all staffing levels. Site staff was supported by a broad group of home office professional and subject matter experts. This yielded a strong and coherent project team that paved the way to success.
The logistical effort required to support a project located at a remote location under difficult access conditions cannot be overestimated. Extensive planning and foresight are required to identify potential issues and corresponding solutions.
The long-term sustainability of post project operations should be a key consideration from the start of the conceptual phase of the project. Issues such as training, supply of equipment and parts, consumables, IT requirements, etc. are areas that need to be addressed.
The three primary goals of the project were to stabilize the foundation of Mosul Dam; provide or improve the equipment, site infrastructure and materials to operate and maintain the dam; and ensure that the workforce was highly skilled with the state of the art equipment, materials and engineering and geologic analyses techniques necessary to maintain the dam