A beginner's guide to physics-informed machine learning and operator learning for geoscientist

Course Description

The purpose of this course is to introduce the geoscience community to emerging machine learning strategies that are better aligned with the nature of geoscientific problems, particularly those governed by physical laws and requiring mappings between fields, defined as functions over spatial, temporal, or spatiotemporal domains. They are typically expressed mathematically through PDEs. These approaches are increasingly important in an era of rapid advances in AI.

Thematically, the course is organized into two major components: PIML and operator learning. Our target audience is the broader geoscience community, not only those who work with PDE-based problems on a daily basis. To support this inclusiveness, each module consists of 1) a preliminary section (lecture only) to ensure everyone shares a basic foundation, 2) the main instructional section (lecture and hands-on practice with Python) where the learning strategies are introduced in depth. On top of this, the course begins with opening remarks and ends with a closing session, providing a complete, well-structured learning experience.

The opening remarks communicate the course motivation, learning objectives, and align participant expectations. The preparation module for PIML will provide a brief review of what PDE-constrained forward and inverse problems are, and classical numerical solvers e.g. finite-difference method. We then explain how deep neural networks (DNNs) work in a purely data-driven supervised learning setup, which serves as a useful reference point for introducing PIML. This also gives us an opportunity to refresh the concepts of backpropagation, particularly automatic differentiation and the chain rule.

Then we move to the instructional section of PIML. We begin with lecture on physics-informed neural networks (PINNs), focusing on how physical laws are embedded into their loss functions to enforce PDE residuals as well as initial and boundary condition constraints. Both forward and inverse problems will be covered. Then we transition into a Python tutorial session, where audiences implement what they have learned using Python code.

After PIML, we move on to operator learning, following a similar structure. We begin with its preparation module, where we revisit examples from computer vision and natural language processing to highlight how conventional DNNs treat data as finite-dimensional vectors. We then review how the internal linear mapping function acts on these feature vectors before applying a nonlinearity. 

Then we move to the instructional section of operator learning. Building on the earlier review of linear mapping function in conventional DNNs, we introduce the concept of linear integral operator. We then present well-known neural operator architectures, including DeepONet and the Fourier neural operator (FNO). Following this, we briefly introduce physics-informed neural operators (PINO), which extend the neural operator framework by incorporating physical constraints. As with the PIML module, the operator learning lectures are followed by a hands-on Python session.

The course concludes with a closing session that synthesizes the key concepts presented, highlights ongoing advances such as foundation-model-style development in the broader field of scientific machine learning and discusses opportunities for applying these techniques to emerging geoscientific problems.

Course Outline

Opening Remarks: Motivation, learning objectives, expectation setting 

Preliminary Section

-PDEs in geoscience & forward/inverse problems recap 

-Classical numerical solvers refresher (finite-difference method)   

- Recap of supervised learning, backpropagation, automatic differentiation & chain rule

Lecture: Physics-informed neural networks     

-PDE-governed forward and inverse problems 

Hands-On Practice: Python tutorial and examples of practical applications Q&A + Discussion 

- Revisit deep neural networks in computer vision and natural language processing 

Lecture: Operator Learning     

-Neural operator concept    

-DeepONet     

- Fourier Neural Operator     

- Physics-Informed Neural Operator

Hands-On Practice: Python tutorial and examples of practical applications

Closing Session: Summary, key takeaways, research outlook (20 min) Q&A + Discussion (30 min)

Participants’ Profile

The course is designed for the broader geoscience community, not only those who work with PDE-based problems on a daily basis. To support this inclusiveness, each module consists of two parts: a preliminary section (lecture only) to ensure everyone shares a basic foundation, followed by the main instructional section (lecture and hands-on practice with Python) where the learning strategies are introduced in depth. On top of this, the course begins with opening remarks and ends with a closing session, enhancing coherence and providing a complete, well-structured learning experience.

Prerequisites

Prior familiarity with (1) numerical solutions of differential equations, e.g., finite-difference method, and (2) automatic differentiation will help participants better understand the nature of PINNs. Prior familiarity with traditional deep learning, especially neural networks such as U-Net, ResNet, and Transformers, will help participants better understand the characteristics of neural operators. However, these are not strict requirements; participants will still be able to follow the course with the support of the preliminary modules. Participants are expected to be familiar with Python coding and with using platforms such as GitLab or similar tools. The course only allocates time for environment setup.

About the Instructor

Dr. Jing Sun is an Assistant Professor in the Intelligent Systems Department at Delft University of Technology, the Netherlands. She completed an industrial Ph.D. in machine learning and applied geophysics at the University of Oslo, co-funded by Viridien (formerly CGG) in Norway. Before joining Delft University of Technology, she worked as a Research Geophysicist in R&D at Shearwater GeoServices, UK.