ITC-CSCC2024 Okinawa (The 39th International Technical Conference on Circuits/Systems, Computers and Communications) ITC-CSCC2024 (The 39th International Technical Conference on Circuits/Systems, Computers and Communications)

Keynote Speakers

Graph Constructions for Machine Learning Applications: New Insights and Algorithms

Presenter

Prof. Antonio Ortega

Prof. Antonio Ortega

  • Department of Electrical and Computer Engineering
    University of Southern California

Abstract

Graphs have long been used in various problems, such as analyzing social networks, machine learning, network protocol optimization, or image processing. In the last few years, a growing body of work has been developed to extend and complement well-known concepts in spectral graph theory, leading to the emergence of Graph Signal Processing (GSP) as a broad research field. In this talk, we summarize recent results that lead to a GSP perspective of machine learning problems. The main observation is that representations of sample data points (e.g., images in a training set) can be used to construct graphs, with nodes representing samples, label information resulting in graph signals, and edge weights capturing the relative positions of samples in feature space. We will first review how this perspective has been used in well-known techniques for label propagation and semi-supervised learning. Then, we will introduce the non-negative kernel regression (NNK) graph construction, describe its properties, and introduce example applications in machine learning areas such as i) model explainability, ii) local interpolative classification, and iii) self-supervised learning.

BIO

Antonio Ortega received his undergraduate and doctoral degrees from the Universidad Politecnica de Madrid, Madrid, Spain and Columbia University, New York, NY, respectively. At Columbia, he was supported by a Fulbright Scholarship. In 1994, he joined the Electrical Engineering department at the University of Southern California (USC), where he is currently a Professor and has served as Associate Chair. He is a Fellow of the IEEE and EURASIP. He was the Editor-in-Chief of the IEEE Transactions of Signal and Information Processing over Networks and will serve as VP of Publications for the IEEE Signal Processing Society from January 2024. He has received several paper awards, including the 2016 Signal Processing Magazine award. His recent research focuses on graph signal processing, machine learning, and multimedia compression. Over 40 PhD students have completed their PhD thesis under his supervision, and his work has led to over 400 publications in international conferences and journals and several patents. He is the author of the book, "Introduction to Graph Signal Processing," published by Cambridge University Press in 2022.

Local All-Pass Image Registration

Presenter

Prof. Thierry Blu

Prof. Thierry Blu

  • Department of Electrical Engineering, National Taiwan University

Abstract

Image alignment/registration is an essential step in applications where two or more images are compared, factoring out the possible geometric distortions between them (e.g., medical imaging, remote sensing, computer vision). We present a novel algorithmic framework for estimating such geometric transformations, by re-interpreting their local effect---shifting---as a signal processing operation: a convolution with an all-pass filter. The key advantage of this reformulation is that no restrictions are placed on the amplitude of the deformation or on the spatial variations of the images. Moreover, by identifying all-pass filtering with a forward-backward filtering relation, our solution to the estimation problem boils down to solving a linear system of equations, which leads to a highly efficient implementation. The effectiveness of this algorithm is demonstrated on a variety of synthetic and real deformations that are found in applications such as image registration and motion estimation. In particular, the Local All-Pass (LAP) algorithm obtains very accurate results for significantly reduced computation time when compared to a selection of image registration algorithms and is very robust to noise corruption. When the transformation has a parametric representation, this robustness can be exploited to deal with large intensity differences between the two images.

BIO

Thierry Blu received the "Diplôme d'ingénieur" from École Polytechnique, France, in 1986 and from Télécom Paris (ENST), France, in 1988. In 1996, he obtained a Ph.D in electrical engineering from ENST for a study on iterated rational filter-banks, applied to wide-band audio coding. Between 1998 and 2007, he was with the Biomedical Imaging Group at the Swiss Federal Institute of Technology (EPFL) in Lausanne, Switzerland. Between 2008 and 2024, he was with the Department of Electronic Engineering (Chinese University of Hong Kong). Since February 2024, he has been a Professor in the Department of Electrical Engineering, National Taiwan University. He was elected Fellow of the IEEE in 2012 for "fundamental contributions to approximation theory in signal and image processing".

Plenary Speakers

Processing-in-DRAM: Opportunities and Design Considerations

Presenter

Prof. Seon Wook Kim

Prof. Seon Wook Kim

  • School of Electrical Engineering, Korea University, Seoul, Korea

Abstract

The required data size and computation throughput of the recently proposed transformer models rapidly increase, thus causing memory bottlenecks to determine the overall system performance. High bandwidth memory (HBM) has been popularly used with GPUs and accelerators to overcome the bottlenecks. However, data transfer from memory to computing devices is still problematic from the viewpoints of computation throughput and power consumption. One of the challenging approaches to resolving the problem uses the "Processing-in-Memory (PIM)" device, especially DRAM, by designing a processing unit inside the device, performing the computation near memory banks, exploiting internal bandwidth, and eliminating the data transfer to a memory controller. Major DRAM makers have introduced the PIM samples, executing all banks' computations simultaneously to maximize the internal DRAM bandwidth to achieve the highest performance. However, their commercialization is still problematic since we must reconsider the interaction between all the architectural layers, from programming models to memory devices, and we would require system component modifications. In this talk, we first review the PIM performance opportunities in the recent transformer models. Then, we discuss the PIM design issues in all the architectural layers and propose possible solutions from recent research targeting PIM commercialization.

BIO

Prof. Seon Wook Kim received his B.S. in Electronics and Computer Engineering from Korea University in 1988, an M.S. in Electrical Engineering from Ohio State University in 1990, and a Ph.D. in Electrical and Computer Engineering from Purdue University in 2001. He was a senior researcher at the Agency for Defense Development and a staff software engineer at Intel. Since Fall 2002, he has been a professor at the School of Electrical and Computer Engineering at Korea University. He has won more than 20 awards due to his teaching and research excellence, including the Haedong Academic Award in 2020. His research interests include compiler construction, microarchitecture, system optimization, and SoC design. He is a senior member of ACM and IEEE.

Approximating and learning 1-Wasserstein Distance with Trees

Presenter

Makoto Yamada

Prof. Makoto Yamada

  • Okinawa Institute of Science and Technology (OIST)

Abstract

Wasserstein distance, which measures the discrepancy between distributions, shows efficacy in various types of natural language processing (NLP) and computer vision (CV) applications. One of the challenges in estimating Wasserstein distance is that it is computationally expensive and does not scale well for many distribution comparison tasks. In this talk, I propose a learning-based approach to approximate the 1-Wasserstein distance with trees. Then, I demonstrate that the proposed approach can accurately approximate the original 1-Wasserstein distance for NLP tasks. Moreover, I introduce the problem of self-supervised learning (SSL) utilizing the 1-Wasserstein distance on a tree structure (a.k.a., Tree-Wasserstein distance (TWD)).

BIO

Makoto Yamada received the Ph.D. degree in statistical science from The Graduate University for Advanced Studies (SOKENDAI, The Institute of Statistical Mathematics), Tokyo, in 2010. Currently, he is an associate professor at Okinawa Institute of Science and Technology (OIST). His research interests include machine learning and its application to biology, natural language processing, and computer vision. He published more than 50 research papers in premium conferences and journals such as NeurIPS, AISTATS, ICML, AAAI, IJCAI, and TPAMI, and won the WSDM 2016 Best Paper Award.