Graph signal processing is a theoretical framework that generalizes classical discrete signal processing from regular domains, such as lines and rectangular lattices, to arbitrary, irregular domains commonly represented by graphs. Different from network science, signal processing on graphs focuses on the interplay between the graph structure and the corresponding signals. The goal of this research is to build a theoretical foundation from the perspective of signal processing to handle practical data analysis tasks. The current work focuses on understanding and formulating such a framework for signal sampling and recovery on graphs.

The input data fed into deep learning systems is usually associated with regular structures. For example, speech signals and natural language have an underlying 1D sequential structure; images have an underlying 2D lattice structure. To take advantage of this regularly structured data, deep learning uses a series of basic operations defined for the regular domain, such as traditional convolution and uniform pooling; however, not all data is contained in regular structures. In urban science, traffic information is associated with road networks; in neuroscience, brain activity is associated with brain connectivity networks; in social sciences, users profile information is associated with social networks. New operations are needed to activate deep learning for irregular data. We design novel graph-based differential operators that handle data associated with irregular structures and also allow end-to-end training. The applications include 3D-skeleton-based action understadning, 3D point cloud learning and social network analysis.

Over the past few decades, signal processing and machine learning tools have been widely generalized from 1D time series to 2D images because of the growth of imaging technologies and huge public demands. Recently, we are experiencing a transition from 2D images to 3D data. Twenty years from now, autonomous driving and virtual/augmented reality might be a part of everyone's daily life. The success of both fields heavily relies on a similar, but new data structure, 3D point clouds. 3D points are irregularly sampled from the surfaces of objects; each point precisely records a position on a surface. The traditional data structures, such as 1D time series and 2D image, naturally have compact discrete representations on a regular grid; however, 3D point clouds are sparsely and irregularly scattered in a 3D continuous space, which cannot be handled by traditional signal processing and machine learning tools. To solve this irregular-data-structure issue, we can introduce a 3D spatial graph to capture the overall shape of scattered 3D points. This 3D spatial graph, which is constructed by connecting neighboring 3D points, can be considered as a warped version of a regular grid. We then can apply techniques developed from data science with graphs to solve many challenges on 3D point cloud, such as 3D data compression, 3D object detection and 3D instance segmentation.

We explore an indirect measurement approach for bridge structural health monitoring that collects sensed information from the dynamic responses of many vehicles travelling over a bridge and then makes extensive use of advanced signal processing techniques to determine information about the state of the bridge.