Streaming Compression of Scientific Data via weak-SINDy

Benjamin P. Russo and M. Paul Laiu, Richard Archibald

In this paper a streaming weak-SINDy algorithm is developed specifically for compressing streaming scientific data. The production of scientific data, either via simulation or experiments, is undergoing an stage of exponential growth, which makes data compression important and often necessary for storing and utilizing large scientific data sets. As opposed to classical ``offline" compression algorithms that perform compression on a readily available data set, streaming compression algorithms compress data ``online" while the data generated from simulation or experiments is still flowing through the system. This feature makes streaming compression algorithms well-suited for scientific data compression, where storing the full data set offline is often infeasible. This work proposes a new streaming compression algorithm, streaming weak-SINDy, which takes advantage of the underlying data characteristics during compression. The streaming weak-SINDy algorithm constructs feature matrices and target vectors in the online stage via a streaming integration method in a memory efficient manner. The feature matrices and target vectors are then used in the offline stage to build a model through a regression process that aims to recover equations that govern the evolution of the data. For compressing high-dimensional streaming data, we adopt a streaming proper orthogonal decomposition (POD) process to reduce the data dimension and then use the streaming weak-SINDy algorithm to compress the temporal data of the POD expansion. We propose modifications to the streaming weak-SINDy algorithm to accommodate the dynamically updated POD basis. By combining the built model from the streaming weak-SINDy algorithm and a small amount of data samples, the full data flow could be reconstructed accurately at a low memory cost, as shown in the numerical tests.

Convergence of weak-SINDy Surrogate Models

Benjamin P. Russo and M. Paul Laiu

In this paper, we give an in-depth error analysis for surrogate models generated by a variant of the Sparse Identification of Nonlinear Dynamics (SINDy) method. We start with an overview of a variety of non-linear system identification techniques, namely, SINDy, weak-SINDy, and the occupation kernel method. Under the assumption that the dynamics are a finite linear combination of a set of basis functions, these methods establish a matrix equation to recover coefficients. We illuminate the structural similarities between these techniques and establish a projection property for the weak-SINDy technique. Following the overview, we analyze the error of surrogate models generated by a simplified version of weak-SINDy. In particular, under the assumption of boundedness of a composition operator given by the solution, we show that (i) the surrogate dynamics converges towards the true dynamics and (ii) the solution of the surrogate model is reasonably close to the true solution. Finally, as an application, we discuss the use of a combination of weak-SINDy surrogate modeling and proper orthogonal decomposition (POD) to build a surrogate model for partial differential equations (PDEs).

Occupation Kernel Hilbert Spaces and the Spectral Analysis of Nonlocal Operators

Joel A. Rosenfeld, Benjamin Russo, and Xiuling Li

This manuscript introduces a space of functions, termed occupation kernel Hilbertspace (OKHS), that operate on collections of signals ratherthan real or complex functions. Tosupport this new definition, an explicit class of OKHSs is given through the consideration of a repro-ducing kernel Hilbert space (RKHS). This space enables the definition of nonlocal operators, suchas fractional order Liouville operators, as well as spectral decomposition methods for correspondingfractional order dynamical systems. In this manuscript, a fractional order DMD routine is presented,and the details of the finite rank representations are given.Significantly, despite the added theoreti-cal content through the OKHS formulation, the resultant computations only differ slightly from thatof occupation kernel DMD methods for integer order systems posed over RKHSs.

Theoretical Foundations for Higher Order Dynamic Mode Decompositions

Joel A. Rosenfeld, Rushikesh Kamalapurkar, and Benjamin P. Russo

Conventionally, data driven identification and control problems for higher order dynamical systems are solved by augmenting the system state by the derivatives of the output to formulate first order dynamical systems in higher dimensions. However, solution of the augmented problem typically requires knowledge of the full augmented state, which requires numerical differentiation of the original output, frequently resulting in noisy signals. This manuscript develops the theory necessary for a direct analysis of higher order dynamical systems using higher order Liouville operators. Fundamental to this theoretical development is the introduction of signal valued RKHSs. Ultimately, it is observed that despite the added abstractions, the necessary computations are remarkably similar to that of first order DMD methods using occupation kernels.

The Occupation Kernel Method for Nonlinear System Identification

Joel A. Rosenfeld, Benjamin Russo, Rushikesh Kamalapurkar, Taylor T. Johnson

This manuscript presents a novel approach to nonlinear system identification leveraging densely defined Liouville operators and a new "kernel" function that represents an integration functional over a reproducing kernel Hilbert space (RKHS) dubbed an occupation kernel. The manuscript thoroughly explores the concept of occupation kernels in the contexts of RKHSs of continuous functions, and establishes Liouville operators over RKHS where several dense domains are found for specific examples of this unbounded operator. The combination of these two concepts allow for the embedding of a dynamical system into a RKHS, where function theoretic tools may be leveraged for the examination of such systems. This framework allows for trajectories of a nonlinear dynamical system to be treated as a fundamental unit of data for nonlinear system identification routine. The approach to nonlinear system identification is demonstrated to identify parameters of a dynamical system accurately, while also exhibiting a certain robustness to noise.

Time-dependent SOLPS-ITER simulations of the tokamak plasma boundary for model predictive control using SINDy*

J.D. Lore, S. De Pascuale, P. Laiu, B. Russo, J.-S. Park, J.M. Park, S.L. Brunton, J.N. Kutz and A.A. Kaptanoglu

Time-dependent SOLPS-ITER simulations have been used to identify reduced models with the sparse identification of nonlinear dynamics (SINDy) method and develop model-predictive control of the boundary plasma state using main ion gas puff actuation. A series of gas actuation sequences are input into SOLPS-ITER to produce a dynamic response in upstream and divertor plasma quantities. The SINDy method is applied to identify reduced linear and nonlinear models for the electron density at the outboard midplane $n^{\text{OMP}}_{\text{e,sep}}$ and the electron temperature at the outer divertor $T^{\text{div}}_{\text{e,sep}}$. Note that $T^{\text{div}}_{\text{e,sep}}$ is not necessarily the peak value of $T_{\text{e}}$ along the divertor. The identified reduced models are interpretable by construction (i.e. not black box), and have the form of coupled ordinary differential equations. Despite significant noise in $T^{\text{div}}_{\text{e,sep}}$, the reduced models can be used to predict the response over a range of actuation levels to a maximum deviation of 0.5% in $n^{\text{OMP}}_{\text{e,sep}}$ and 5%–10% in $T^{\text{div}}_{\text{e,sep}}$ for the cases considered. Model retraining using time history data triggered by a preset error threshold is also demonstrated. A model predictive control strategy for nonlinear models is developed and used to perform feedback control of a SOLPS-ITER simulation to produce a setpoint trajectory in $n^{\text{OMP}}_{\text{e,sep}}$ using the integrated plasma simulator framework. The developed techniques are general and can be applied to time-dependent data from other boundary simulations or experimental data. Ongoing work is extending the approach to model identification and control for divertor detachment, which will present transient nonlinear behavior from impurity seeding, including realistic latency and synthetic diagnostic signals derived from the full SOLPS-ITER output.

Bayesian inversion and the Tomita--Takesaki modular group

Luca Giorgetti, Arthur J. Parzygnat, Alessio Ranallo, Benjamin P. Russo

We show that conditional expectations, optimal hypotheses, disintegrations, and adjoints of unital completely positive maps, are all instances of Bayesian inverses. We study the existence of the latter by means of the Tomita--Takesaki modular group and we provide extensions of a theorem of Takesaki as well as a theorem of Accardi and Cecchini to the setting of not necessarily faithful states on finite-dimensional $C^*$-algebras.

Motion Tomography via Occupation Kernels

Benjamin P. Russo, Rushikesh Kamalapurkar, Dongsik Chang, and Joel A. Rosenfeld

Published in Journal of Computational Dynamics, Volume 9, Issue 1, January 2022

The goal of motion tomography is to recover a description of a vector flow field using information on the trajectory of a sensing unit. In this paper, we develop a predictor corrector algorithm designed to recover vector flow fields from trajectory data with the use of occupation kernels developed by Rosenfeld et al.. Specifically, we use the occupation kernels as an adaptive basis; that is, the trajectories defining our occupation kernels are iteratively updated to improve the estimation on the next stage. Initial estimates are established, then under mild assumptions, such as relatively straight trajectories, convergence is proven using the Contraction Mapping Theorem. We then compare to the established method by Chang et al. by defining a set of error metrics. We found that for simulated data, which provides a ground truth, our method offers a marked improvement and that for a real-world example we have similar results to the established method.

Spectra for Toeplitz Operators Associated with a Constrained Subalgebra

Christopher Felder, Douglas T. Pfeffer, Benjamin P. Russo

Published in Integral Equations and Operator Theory volume 94, Article number: 24, 2022

A two-point algebra is a set of bounded analytic functions on the unit disk that agree at two distinct points $a,b\in \mathbb{D}$. This algebra serves as a multiplier algebra for the family of Hardy Hilbert spaces $H^2_t:=\{f\in H^2 : f(a) = tf(b)\}$, where $t\in \mathbb{C}\cup \{\infty\}$. We show that various spectra of certain Toeplitz operators acting on these spaces are connected.

Non-commutative disintegrations: existence and uniqueness in finite dimensions

Arthur J. Parzygnat, Benjamin P. Russo

Accepted for Publication

We utilize category theory to define non-commutative disintegrations, regular conditional probabilities, and optimal hypotheses for finite-dimensional $C^*$-algebras. In the process, we introduce a notion of a.e. equivalence for positive maps and show that the category of $C^*$-algebras and a.e. equivalence classes of 2-positive unital maps forms a category. A related result holds for positive unital maps on von Neumann algebras. In the special case of a finite-dimensional commutative $C^*$-algebra, this reproduces the usual notions of a.e. equivalence and a disintegration of a probability measure over another measure consistent with a probability-preserving function. Similar to the commutative (measure-theoretic) case, disintegrations are unique almost everywhere whenever they exist. However, in contrast to the commutative case, there are many instances where such disintegrations do not exist. We show a certain separability condition on the density matrices representing the states is necessary and sufficient for the existence and uniqueness of such disintegration on finite-dimensional $C^*$-algebras. Physically, tracing out degrees of freedom from the environment of a quantum system is one example of a state-preserving *-homomorphism and the disintegration is the optimal reversal of this procedure. Finally, we discuss some implications for quantum measurement.

A non-commutative Bayes' theorems

Arthur J. Parzygnat, Benjamin P. Russo

Published in Linear Algebra and Applications, Volume 644, July 2022

Using a diagrammatic reformulation of Bayes' theorem, we provide a necessary and sufficient condition for the existence of Bayesian inference in the setting of finite-dimensional $C^∗$-algebras. In other words, we prove an analogue of Bayes' theorem in the joint classical and quantum context. Our analogue is justified by recent advances in categorical probability theory, which have provided an abstract formulation of the classical Bayes' theorem. In the process, we further develop non-commutative almost everywhere equivalence and illustrate its important role in non-commutative Bayesian inversion. The construction of such Bayesian inverses, when they exist, involves solving a positive semidefinite matrix completion problem for the Choi matrix. This gives a solution to the open problem of constructing Bayesian inversion for completely positive unital maps acting on density matrices that do not have full support. We illustrate how the procedure works for several examples relevant to quantum information theory.

Liouville Operators over the Hardy Space

Benjamin P. Russo and Joel A. Rosenfeld

Published in Journal of Mathematical Analysis and Applications, Volume 508, Issue 2, 15 April 2022

The role of Liouville operators in the study of dynamical systems through the use of occupation measures have been an active area of research in control theory over the past decade. This manuscript investigates Liouville operators over the Hardy space, which encode complex ordinary differential equations in an operator over a reproducing kernel Hilbert space.

Occupation Kernels and Densely Defined Liouville Operators for System Identification

Joel A. Rosenfeld; Rushikesh Kamalapurkar; Benjamin Russo; Taylor T. Johnson

2019 IEEE 58th Conference on Decision and Control (CDC)

This manuscript introduces the concept of Liouville operators and occupation kernels over reproducing kernel Hilbert spaces (RKHSs). The combination of these two concepts allow for the embedding of a dynamical system into a RKHS, where function theoretic tools may be leveraged for the examination of such systems. These tools are then turned toward the problem of system identification where an inner product formula is developed to provide constraints on the parameters in a system identification setting. This system identification routine is validated through several numerical experiments, where each experiment examines various contributions to the parameter identification error via numerical integration methods and parameters for the kernel functions themselves.

The Mittag Leffler Reproducing Kernel Hilbert Spaces of Entire and Analytic Functions

Joel Rosenfeld, Benjamin Russo, and Warren E. Dixon

Published in Journal of Mathematical Analysis and Applications, Volume 463, Issue 2, 15 July 2018, Pages 576-592

This paper investigates the function theoretic properties of two reproducing kernel functions based on the Mittag-Leffler function that are related through a composition. Both spaces provide one parameter generalizations of the traditional Bargmann-Fock space. In particular, the Mittag-Leffler space of entire functions yields many similar properties to the Bargmann-Fock space, and several results are demonstrated involving zero sets and growth rates. The second generalization, the Mittag-Leffler space of the slitted plane, is a reproducing kernel Hilbert space (RKHS) of functions for which Caputo fractional differentiation and multiplication by $z^q$ (for $q>0$) are densely defined adjoints of one another.

Lifting Commuting 3-Isometric Tuples

Published in Operators and Matrices, 2017, Volume 11, Number 2, pp 397–433

An operator $T$ is called a $3$-isometry if there exist operators $B_1(T^*,T)$ and $B_2(T^*,T)$ such that $$Q(n) = T^{*n}T^{n} = 1 + nB_1(T^*,T)+n^2B_2(T^*,T)$$ for all natural numbers $n$. An operator $J$ is a Jordan operator of order $2$ if $J=U+N$ where $U$ is unitary, $N$ is nilpotent order $2$, and $U$ and $N$ commute. An easy computation shows that $J$ is a $3$-isometry and that the restrictions of $J$ to an invariant subspace is also a $3$-isometry. Those $3$-isometries which are the restriction of a Jordan operator to an invariant subspace can be identified, using the theory of completely positive maps, in terms of a positivity condition on the operator pencil $Q(s)$. In this article, we establish the analogous result in the multi-variable setting and show, by modifying an example of Choi, that an additional hypothesis is necessary. Lastly we discuss the joint spectrum of sub-Jordan tuples and derive the results of $3$-symmetric operators as a corrolary.

A Lifting Theorem for 3-Isometries

Scott McCullough and Ben Russo

Published in Integral Equations and Operator Theory, January 2016, Volume 84, Issue 1, pp 69–87

An operator $T$ on Hilbert space is a $3$-isometry if there exist operators $B_1$ and $B_2$ such that $$T^{*n}T^n = I+nB_1 +n^2 B_2.$$ An operator $J$ is a Jordan operator if it the sum of a unitary $U$ and nilpotent $N$ of order two which commute. If $T$ is a $3$-isometry and $c>0$, then $I-c^{-2} B_2 + sB_1 + s^2B_2$ is positive semidefinite for all real $s$ if and only if it is the restriction of a Jordan operator $J = U + N$ with the norm of $N$ at most $c$. As a corollary, an analogous result for $3$-symmetric operators, due to Helton and Agler, is recovered.