SPECFEM3D Globe

SPECFEM3D_GLOBE simulates global and regional (continental-scale) seismic wave propagation. 

For more information, visit

• https://geodynamics.org/cig/software/specfem3d_globe/

We thank the Computational Infrastructure for Geodynamics (CIG) for hosting SPECFEM3D GLOBE which is supported by the National Science Foundation award NSF-0949446 and NSF-1550901.

For more information about the Computational Infrastructure for Geodynamics (CIG), visit

• https://geodynamics.org/cig/

For more information on the ROCm™ open software platform and access to an active community discussion on installing, configuring, and using ROCm, please visit the ROCm web pages at www.AMD.com/ROCm and the ROCm Community Forum.

 

Single-Node Server Requirements

---

 

CPUs / GPUs	Operating Systems	ROCm Driver	Container Runtimes
X86_64 CPU(s) AMD Instinct MI200 GPU(s) AMD Instinct MI100 GPU(s) Radeon Instinct MI50(S)	Ubuntu 18.04 Centos 8.3 RHEL 8.2	ROCm version 4.5 compatibility	Docker Engine Singularity

Note: The SPECFEM3D Global application container assumes that the server contains the required x86-64 CPU(s) and at least one of the listed AMD GPUs. Also, the server must have one of the required operating systems and the listed ROCm driver version installed to run the Docker container. The server must also have a Docker Engine installed to run the container. Please visit the Docker Engine install web site at https://docs.docker.com/engine/install/ to install the latest Docker Engine for the operating system installed on the server. If Singularity use is planned, please visit https://sylabs.io/docs/ for the latest Singularity install documentation.

Please visit https://rocmdocs.amd.com/en/latest/Installation_Guide/Installation-Guide.html for ROCm installation procedures and validation checks.

 

Licensing Information

---

Your use of this application is subject to the terms of the GNU General Public License, version 3 or (at your option) any later version (https://www.gnu.org/licenses/gpl-3.0.html). Source code form will be made available upon request. By accessing and using this application, you are agreeing to fully comply with the terms of this license. If you do not agree to the terms of this license, do not access or use this application.
 
Additional third-party content in this container may be subject to additional licenses and restrictions. Third-party content is licensed to you directly by the third party that owns the content pursuant to the third-party license terms included with such content, and is not licensed to you by AMD. ALL LINKED THIRD-PARTY CONTENT IS MADE AVAILABLE BY AMD “AS IS” WITHOUT A WARRANTY OF ANY KIND. USE OF SUCH THIRD-PARTY CONTENT IS DONE AT YOUR SOLE DISCRETION AND UNDER NO CIRCUMSTANCES WILL AMD BE LIABLE TO YOU FOR ANY THIRD-PARTY CONTENT. YOU ASSUME ALL RISK AND ARE SOLELY RESPONSIBLE FOR ANY DAMAGES THAT MAY ARISE FROM YOUR USE OF THIRD-PARTY CONTENT.

 

References

---

We use SPECFEM3D GLOBE FROM SOURCE ( Komatitsch and Tromp, 2002; Komatitsch and Tromp, 2002; Komatitsch et al., 9999) published under the GPL 3 license.

We thank the Computational Infrastructure for Geodynamics (http://geodynamics.org) which is funded by the National Science Foundation under awards EAR-0949446 and EAR-1550901.

Komatitsch, D.; Vilotte, J.-P.; Tromp, J.; Afanasiev, M.; Bozdag, E.; Charles, J.; Chen, M.; Goddeke, D.; Hjorleifsdottir, V.; Labarta, J.; Le Goff, N.; Le Loher, P.; Liu, Q.; Maggi, A.; Martin, R.; McRitchie, D.; Messmer, P.; Michea, D.; Nissen-Meyer, T.; Peter, D.; Rietmann, M.; de Andrade, S.; Savage, B.; Schuberth, B.; Siemenski, A.; Strand, L.; Tape, C.; Xie, Z.; Zhu, H. (9999), SPECFEM3D GLOBE [software], Computational Infrastructure for Geodynamics, doi: GITHASH8, url: https://geodynamics.org

 

Running Containers

---

Start the Docker container using the following command:

docker run --rm -it --device=/dev/dri --device=/dev/kfd --security-opt seccomp=unconfined amdih/specfem3d_globe:1ee10977 /bin/bash

 

Running Application

---

Please refer to SpecFEM3D Globe documentation for working with specfem3d globe.

There is a convenience benchmark script included. This benchmark problem requires a dual socket server with 6 GPUs and at least 70 GB of free storage space. After starting the container, you may execute it as follows

export OMPI_ALLOW_RUN_AS_ROOT=1
export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1
benchmark global_s362ani_shakemovie -o /tmp/out

This will write the output files to /tmp/out.

Other examples

You may also navigate into EXAMPLES directory of specfem3d globe. Please note that different examples may have different hardware requirements, for example, some examples may require 6 GPUs.

Consider running a single rank example

cd /opt/specfem3d_globe/EXAMPLES/regional_Greece_small
export OMPI_ALLOW_RUN_AS_ROOT=1
export OMPI_ALLOW_RUN_AS_ROOT_CONFIRM=1
./run_this_example.sh

Please note that specfem3d requires recompilation of the solver given input files. The above script will compile the mesher and solver and run the analysis

Singularity support

This section assumes that an up-to-date version of Singularity is installed on your system and properly configured for your system. Please consult with your system administrator or view official Singularity documentation.

Pull and convert docker image to singularity image format:

singularity pull specfem3d_globe.sif docker://amdih/specfem3d_globe: 1ee10977

1. Run the singularity container

singularity run --writable-tmpfs singularity_globe.sif /bin/bash

You can follow steps from preceding section to run an existing example within the container, for example, you may run an example:

cd /opt/specfem3d_globe/EXAMPLES/regional_Greece_small
./run_this_example.sh

 

Disclaimer

---

The information contained herein is for informational purposes only, and is subject to change without notice. In addition, any stated support is planned and is also subject to change. While every precaution has been taken in the preparation of this document, it may contain technical inaccuracies, omissions and typographical errors, and AMD is under no obligation to update or otherwise correct this information. Advanced Micro Devices, Inc. makes no representations or warranties with respect to the accuracy or completeness of the contents of this document, and assumes no liability of any kind, including the implied warranties of noninfringement, merchantability or fitness for particular purposes, with respect to the operation or use of AMD hardware, software or other products described herein. No license, including implied or arising by estoppel, to any intellectual property rights is granted by this document. Terms and limitations applicable to the purchase or use of AMD’s products are as set forth in a signed agreement between the parties or in AMD's Standard Terms and Conditions of Sale.

 

Notices and Attribution

---

© 2023 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, Instinct, Radeon Instinct, ROCm, and combinations thereof are trademarks of Advanced Micro Devices, Inc.

Docker and the Docker logo are trademarks or registered trademarks of Docker, Inc. in the United States and/or other countries. Docker, Inc. and other parties may also have trademark rights in other terms used herein. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.

All other trademarks and copyrights are property of their respective owners and are only mentioned for informative purposes.