MENU
  • Scitation Home
SUBMIT YOUR ARTICLE

Thank you

For your interest in Chaos: An Interdisciplinary Journal of Nonlinear Science

Check for updates on crossmark
Prev Next
No Access Submitted: 01 July 2017 Accepted: 25 September 2017 Published Online: 09 October 2017

  • Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator

Chaos 27, 103111 (2017); https://doi.org/10.1063/1.4993854
Qianxiao Li1,a), Felix Dietrich2, Erik M. Bollt3, and Ioannis G. Kevrekidis4,b),c)
more...View Affiliations
View Contributors
  • Qianxiao Li
  • Felix Dietrich
  • Erik M. Bollt
  • Ioannis G. Kevrekidis
ABSTRACT
Numerical approximation methods for the Koopman operator have advanced considerably in the last few years. In particular, data-driven approaches such as dynamic mode decomposition (DMD)51 and its generalization, the extended-DMD (EDMD), are becoming increasingly popular in practical applications. The EDMD improves upon the classical DMD by the inclusion of a flexible choice of dictionary of observables which spans a finite dimensional subspace on which the Koopman operator can be approximated. This enhances the accuracy of the solution reconstruction and broadens the applicability of the Koopman formalism. Although the convergence of the EDMD has been established, applying the method in practice requires a careful choice of the observables to improve convergence with just a finite number of terms. This is especially difficult for high dimensional and highly nonlinear systems. In this paper, we employ ideas from machine learning to improve upon the EDMD method. We develop an iterative approximation algorithm which couples the EDMD with a trainable dictionary represented by an artificial neural network. Using the Duffing oscillator and the Kuramoto Sivashinsky partical differential equation as examples, we show that our algorithm can effectively and efficiently adapt the trainable dictionary to the problem at hand to achieve good reconstruction accuracy without the need to choose a fixed dictionary a priori. Furthermore, to obtain a given accuracy, we require fewer dictionary terms than EDMD with fixed dictionaries. This alleviates an important shortcoming of the EDMD algorithm and enhances the applicability of the Koopman framework to practical problems.

ACKNOWLEDGMENTS

The work of I.G.K. was partially supported by DARPA-MoDyL (HR0011-16-C-0116) and by the U.S. National Science Foundation (ECCS-1462241). I.G.K. and F.D. are grateful for the hospitality and support of the IAS-TUM. F.D. is also grateful for the support from the TopMath Graduate Center of TUM Graduate School at the Technical University of Munich, Germany, and from the TopMath Program at the Elite Network of Bavaria. E.M.B. thanks the Army Research Office (N68164-EG) and the Office of Naval Research (N00014-15-1-2093). Q.L. is grateful for the support of the Agency for Science, Technology and Research, Singapore.

REFERENCES

  1. 1. I. Mezić and A. Banaszuk, “ Comparison of systems with complex behavior,” Physica D 197, 101 (2004). https://doi.org/10.1016/j.physd.2004.06.015, Google ScholarCrossref, ISI
  2. 2. I. Mezić, “ Spectral properties of dynamical systems, model reduction and decompositions,” Nonlinear Dyn. 41, 309–325 (2005). https://doi.org/10.1007/s11071-005-2824-x, Google ScholarCrossref, ISI
  3. 3. B. O. Koopman, “ Hamiltonian systems and transformation in Hilbert space,” Proc. Natl. Acad. Sci. U.S.A. 17, 315–318 (1931). https://doi.org/10.1073/pnas.17.5.315, Google ScholarCrossref
  4. 4. J. v. Neumann, “ Zur operatorenmethode in der klassischen mechanik,” Ann. Math. 33, 587–642 (1932). https://doi.org/10.2307/1968537, Google ScholarCrossref
  5. 5. P. R. Halmos and J. von Neumann, “ Operator methods in classical mechanics, II,” Ann. Math. 43, 332–350 (1942). https://doi.org/10.2307/1968872, Google ScholarCrossref
  6. 6. P. R. Halmos, P. R. Halmos, P. R. Halmos, H. Mathématicien, P. R. Halmos, and H. Mathematician, Introduction to Hilbert Space and the Theory of Spectral Multiplicity ( Chelsea Publishing Company, Chelsea New York, 1957). Google Scholar
  7. 7. C. W. Rowley, I. Mezić, S. Bagheri, P. Schlatter, and D. S. Henningson, “ Spectral analysis of nonlinear flows,” J. Fluid Mech. 641, 115–127 (2009). https://doi.org/10.1017/S0022112009992059, Google ScholarCrossref, ISI
  8. 8. M. Budišić, R. Mohr, and I. Mezić, “ Applied Koopmanism,” Chaos 22, 047510 (2012). https://doi.org/10.1063/1.4772195, Google ScholarScitation, ISI
  9. 9. M. O. Williams, C. W. Rowley, I. Mezić, and I. G. Kevrekidis, “ Data fusion via intrinsic dynamic variables: An application of data-driven koopman spectral analysis,” Europhys. Lett. 109, 40007 (2015). https://doi.org/10.1209/0295-5075/109/40007, Google ScholarCrossref
  10. 10. D. Giannakis, J. Slawinska, and Z. Zhao, “ Spatiotemporal feature extraction with data-driven Koopman operators,” J. Mach. Learn. Res. Proc. 44, 103–115 (2015). Google Scholar
  11. 11. S. L. Brunton, B. W. Brunton, J. L. Proctor, and J. N. Kutz, “ Koopman invariant subspaces and finite linear representation of nonlinear dynamical systems for control,” PloS One 11, e0150171 (2016). https://doi.org/10.1371/journal.pone.0150171, Google ScholarCrossref
  12. 12. M. Korda and I. Mezić, “ Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control,” e-print arXiv:1703.10112. Google Scholar
  13. 13. I. Mezić, “ Analysis of fluid flows via spectral properties of the Koopman operator,” Annu. Rev. Fluid Mech. 45, 357–378 (2013). https://doi.org/10.1146/annurev-fluid-011212-140652, Google ScholarCrossref, ISI
  14. 14. A. S. Sharma, I. Mezić, and B. J. McKeon, “ Correspondence between koopman mode decomposition, resolvent mode decomposition, and invariant solutions of the Navier-Stokes equations,” Phys. Rev. Fluids 1, 032402 (2016). https://doi.org/10.1103/PhysRevFluids.1.032402, Google ScholarCrossref
  15. 15. M. Georgescu and I. Mezić, “ Building energy modeling: A systematic approach to zoning and model reduction using koopman mode analysis,” Energy Build. 86, 794–802 (2015). https://doi.org/10.1016/j.enbuild.2014.10.046, Google ScholarCrossref
  16. 16. H. Wu, F. Nüske, F. Paul, S. Klus, P. Koltai, and F. Noć, “ Variational koopman models: Slow collective variables and molecular kinetics from short off-equilibrium simulations,” J. Chem. Phys. 146, 154104 (2017). https://doi.org/10.1063/1.4979344, Google ScholarScitation, ISI
  17. 17. A. Mauroy, I. Mezić, and J. Moehlis, “ Isostables, isochrons, and koopman spectrum for the action–angle representation of stable fixed point dynamics,” Physica D 261, 19–30 (2013). https://doi.org/10.1016/j.physd.2013.06.004, Google ScholarCrossref
  18. 18. D. Giannakis, “ Data-driven spectral decomposition and forecasting of ergodic dynamical systems,” Applied and Computational Harmonic Analysis (to be published). Google Scholar
  19. 19. S. Klus, N. Peter, K. Peter, W. Hao, K. Ioannis, S. Christof, and N. Frank, “ Data-driven model reduction and transfer operator approximation,” J. Nonlinear Sci. (unpublished). Google Scholar
  20. 20. S. L. Brunton, J. L. Proctor, and J. N. Kutz, “ Discovering governing equations from data by sparse identification of nonlinear dynamical systems,” Proc. Natl. Acad. Sci. U.S.A. 113, 3932–3937 (2016). https://doi.org/10.1073/pnas.1517384113, Google ScholarCrossref, ISI
  21. 21. P. J. Schmid, “ Dynamic mode decomposition of numerical and experimental data,” J. Fluid Mech. 656, 5–28 (2010). https://doi.org/10.1017/S0022112010001217, Google ScholarCrossref, ISI
  22. 22. M. O. Williams, C. W. Rowley, and I. G. Kevrekidis, “ A kernel approach to data-driven koopman spectral analysis,” preprint arXiv:1411.2260 (2014). Google Scholar
  23. 23. M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, “ A data–driven approximation of the koopman operator: Extending dynamic mode decomposition,” J. Nonlinear Sci. 25, 1307–1346 (2015). https://doi.org/10.1007/s00332-015-9258-5, Google ScholarCrossref
  24. 24. M. Korda and I. Mezić, “ On convergence of extended dynamic mode decomposition to the koopman operator,” e-print arXiv:1703.04680v1. Google Scholar
  25. 25. R. K. Singh and J. S. Manhas, Composition Operators on Function Spaces ( Elsevier, 1993), Vol. 179. Google Scholar
  26. 26. C. W. Rowley, see http://online.kitp.ucsb.edu/online/transturb-c17/rowley/ for “ Data-driven methods for identifying nonlinear models of fluid flows” (2017). Google Scholar
  27. 27. D. L. Donoho, “ Compressed sensing,” IEEE Trans. Inf. Theory 52, 1289–1306 (2006). https://doi.org/10.1109/TIT.2006.871582, Google ScholarCrossref, ISI
  28. 28. K. Karhunen, Über lineare Methoden in der Wahrscheinlichkeitsrechnung ( Universitat Helsinki, 1947), Vol. 37. Google Scholar
  29. 29. M. Loeve, Probability Theory, Vol. II, Graduate Texts in Mathematics ( Springer-Verlag New York, 1978), Vol. 46, p. 387. Google ScholarCrossref
  30. 30. A. N. Tikhonov, A. V Goncharsky, V. V. Stepanov, and A. G. Yagola, Numerical Methods for the Solution of Ill-Posted Problems (Springer Science and Business Media, 2013), Vol. 328. Google Scholar
  31. 31. A. Y. Ng, “ Feature selection, l 1 vs. l 2 regularization, and rotational invariance,” in Proceedings of the Twenty-First International Conference on Machine Learning (ACM, 2004), p. 78. Google ScholarCrossref
  32. 32. G. H. Golub and C. F. van Loan, Matrix Computations ( JHU Press, 2012), Vol. 3. Google Scholar
  33. 33. S. Ruder, “ An overview of gradient descent optimization algorithms,” e-print arXiv:1609.04747. Google Scholar
  34. 34. P. Constantin, C. Foias, B. Nicolaenko, and R. Temam, Integral Manifolds and Inertial Manifolds for Dissipative Partial Differential Equations ( Springer Science & Business Media, 2012), Vol. 70. Google Scholar
  35. 35. M. Jolly, I. Kevrekidis, and E. Titi, “ Approximate inertial manifolds for the Kuramoto-Sivashinsky equation: Analysis and computations,” Physica D 44, 38–60 (1990). https://doi.org/10.1016/0167-2789(90)90046-R, Google ScholarCrossref, ISI
  36. 36. H. Kwakernaak and R. Sivan, Linear Optimal Control Systems ( Wiley-Interscience New York, 1972), Vol. 1. Google Scholar
  37. 37. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems ( 2012), pp. 1097–1105. Google Scholar
  38. 38. Y. LeCun, Y. Bengio et al., “ Convolutional networks for images, speech, and time series,” in The Handbook of Brain Theory and Neural Networks (1995), Vol. 3361, p. 1995. Google Scholar
  39. 39. Y. LeCun, K. Kavukcuoglu, and C. Farabet, “ Convolutional networks and applications in vision,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE, 2010), pp. 253–256. Google ScholarCrossref
  40. 40. Y. LeCun, Y. Bengio, and G. Hinton, “ Deep learning,” Nature 521, 436–444 (2015). https://doi.org/10.1038/nature14539, Google ScholarCrossref, ISI
  41. 41. F. Bach and E. Moulines, “ Non-strongly-convex smooth stochastic approximation with convergence rate o (1/n),” in Advances in Neural Information Processing Systems (2013) pp. 773–781. Google Scholar
  42. 42. R. A. DeVore, “ Nonlinear approximation,” Acta Numer. 7, 51–150 (1998). https://doi.org/10.1017/S0962492900002816, Google ScholarCrossref
  43. 43. T. Sauer, J. A. Yorke, and M. Casdagli, “ Embedology,” J. Stat. Phys. 65, 579–616 (1991). https://doi.org/10.1007/BF01053745, Google ScholarCrossref, ISI
  44. 44. A. Kolmogoroff, “ Über die analytischen methoden in der wahrscheinlichkeitsrechnung,” Math. Ann. 104, 415–458 (1931). https://doi.org/10.1007/BF01457949, Google ScholarCrossref
  45. 45. J. H. Tu, C. W. Rowley, D. M. Luchtenburg, S. L. Brunton, and J. N. Kutz, “ On dynamic mode decomposition: Theory and applications,” J. Comput. Dyn. 1, 391–421 (2014). https://doi.org/10.3934/jcd.2014.1.391, Google ScholarCrossref
  1. © 2017 Author(s). Published by AIP Publishing.

Article Metrics

Views
3,744
Citations
Crossref 149
Web of Science
ISI 133

Altmetric

Please Note: The number of views represents the full text views from December 2016 to date. Article views prior to December 2016 are not included.