250081 VU Tensor Methods for Data Science and Scientific Computing (2024W)

This course is a rigorous introduction to several key techniques for the low-rank approximation of tensors (multidimensional arrays), aimed at providing students with basic knowledge and ample opportunity for starting own research in the area. Possible applications, which will be discussed in the course, are associated with such areas as data science and machine learning, quantitative neuroscience, spectroscopy, psychometrics, arithmetic complexity and data compression. Some of the most illustrative applications, however, belong to the field of scientific computing.

The course will first cover the canonical polyadic, Tucker and tensor-train decompositions of multidimensional arrays from a linear-algebraic perspective and then focus on the use of low-rank tensor decompositions in computational mathematics. In the second part, the course will focus on the tensor-train (TT) decomposition, originally developed under the name of matrix product states (MPS) in computational quantum physics. This tensor decomposition appears naturally — as a representation of functions — from low-rank refinement in the construction of finite-element approximations. That is crucial in the context of PDE problems, in which the low-rank approximation of functions, depending on their regularity, will be analyzed and state-of-the-art methods for preconditioning and solving optimality equations (linear systems) will be covered (including the construction, implementation and numerical analysis of such methods). Homework assignments will involve theoretical and implementation tasks. For implementation tasks, every student can use any programming language or environment and the lecturer will present solutions in the form of Jupyter notebooks in Julia.

***
The course spotlights the interplay of two areas of modern applied mathematics:
* low-rank approximation and analysis of abstract data represented by multi-dimensional arrays
and
* adaptive numerical methods for solving PDE problems.

In psychometrics, signal processing, image processing and data mining, low-rank tensor decompositions have been studied as a way of formally generalizing the notion of rank from matrices to higher-dimensional arrays (tensors). Several such generalizations have been proposed, including the canonical polyadic (CP) and Tucker decompositions and the tensor-SVD, with the primary motivation of analyzing, interpreting and compressing datasets. In this context, data are often thought of as parametrizations of images, video, social networks or collections of interconnected texts; on the other hand, data representing functions (which often occur in computational mathematics) are remarkable for the possibility of precise analysis.

The tensor-train (TT) and the more general hierarchical Tucker decompositions were developed in the community of numerical mathematics, more recently and with particular attention to PDE problems. In fact, exactly the same and very similar representations had long been used for the numerical simulation of many-body quantum systems by computational chemists and physicists under the names of «matrix-product states» (MPS) and «multilayer multi-configuration time-dependent Hartree». These low-rank tensor decompositions are based on subspace approximation, which can be performed adaptively and iteratively, in a multilevel fashion. In a broader context of PDE problems, this leads to numerical methods that are formally based on generic discretizations but effectively operate on adaptive, data-driven discretizations constructed «online», in the course of computation. In several settings, such methods achieve the accuracy of sophisticated problem-specific methods.