No Access Submitted: 29 October 2019 Accepted: 11 March 2020 Published Online: 03 April 2020
Chaos 30, 043108 (2020); https://doi.org/10.1063/1.5133725
Data mining is routinely used to organize ensembles of short temporal observations so as to reconstruct useful, low-dimensional realizations of an underlying dynamical system. In this paper, we use manifold learning to organize unstructured ensembles of observations (“trials”) of a system’s response surface. We have no control over where every trial starts, and during each trial, operating conditions are varied by turning “agnostic” knobs, which change system parameters in a systematic, but unknown way. As one (or more) knobs “turn,” we record (possibly partial) observations of the system response. We demonstrate how such partial and disorganized observation ensembles can be integrated into coherent response surfaces whose dimension and parametrization can be systematically recovered in a data-driven fashion. The approach can be justified through the Whitney and Takens embedding theorems, allowing reconstruction of manifolds/attractors through different types of observations. We demonstrate our approach by organizing unstructured observations of response surfaces, including the reconstruction of a cusp bifurcation surface for hydrogen combustion in a continuous stirred tank reactor. Finally, we demonstrate how this observation-based reconstruction naturally leads to informative transport maps between the input parameter space and output/state variable spaces.
This work was partially funded by the National Science Foundation (NSF), the Defense Advanced Research Projects Agency (DARPA) (I.G.K. and F.D.), the SNSF (Grant No. P2EZP2_168833) (M.K.), and the Army Research Office (ARO) (I.G.K., E.M.B.) and Office of Naval Research (ONR) (E.M.B.). Discussions with Professor J. Guckenheimer are gratefully acknowledged.
  1. 1. E. J. Doedel, Congr. Numer. 30, 25 (1981). Google Scholar
  2. 2. E. J. Doedel, T. F. Fairgrieve, B. Sandstede, A. R. Champneys, Y. A. Kuznetsov, and X. Wang (2007), see http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.423.2590. Google Scholar
  3. 3. A. Dhooge, W. Govaerts, and Y. A. Kuznetsov, ACM Trans. Math. Softw. 29, 141 (2003). https://doi.org/10.1145/779359.779362, Google ScholarCrossref
  4. 4. R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker, Proc. Natl. Acad. Sci. U.S.A. 102, 7426 (2005). https://doi.org/10.1073/pnas.0500334102, Google ScholarCrossref
  5. 5. T. Sauer, Phys. Rev. Lett. 72, 3811 (1994). https://doi.org/10.1103/PhysRevLett.72.3811, Google ScholarCrossref
  6. 6. O. Yair, R. Talmon, R. R. Coifman, and I. G. Kevrekidis, Proc. Natl. Acad. Sci. U.S.A. 114, E7865 (2017). https://doi.org/10.1073/pnas.1620045114, Google ScholarCrossref
  7. 7. S. L. Brunton, J. L. Proctor, and J. N. Kutz, Proc. Natl. Acad. Sci. U.S.A. 113, 3932 (2016). https://doi.org/10.1073/pnas.1517384113, Google ScholarCrossref
  8. 8. B. Moore, IEEE Trans. Automat. Contr. 26, 17 (1981). https://doi.org/10.1109/TAC.1981.1102568, Google ScholarCrossref
  9. 9. F. Takens, in Dynamical Systems and Turbulence, Warwick 1980 (Springer, 1981), pp. 366–381. Google Scholar
  10. 10. T. Berry and T. Sauer, Appl. Comput. Harmon. Anal. 40, 439–469 (2015). Google ScholarCrossref
  11. 11. M. Belkin and P. Niyogi, Neural Comput. 15, 1373 (2003). https://doi.org/10.1162/089976603321780317, Google ScholarCrossref
  12. 12. C. J. Dsilva, R. Talmon, R. R. Coifman, and I. G. Kevrekidis, Appl. Comput. Harmon. Anal. 44, 759 (2018). https://doi.org/10.1016/j.acha.2015.06.008, Google ScholarCrossref
  13. 13. T. Berry and J. Harlim, Appl. Comput. Harmon. Anal. 45, 84 (2018). https://doi.org/10.1016/j.acha.2016.08.005, Google ScholarCrossref
  14. 14. F. R. K. Chung, Spectral Graph Theory (American Mathematical Society, 1996). Google ScholarCrossref
  15. 15. B. Nadler, S. Lafon, R. R. Coifman, and I. G. Kevrekidis, Appl. Comput. Harmon. Anal. 21, 113 (2006). https://doi.org/10.1016/j.acha.2005.07.004, Google ScholarCrossref
  16. 16. T. Shnitzer, R. Talmon, and J.-J. Slotine, IEEE Trans. Signal Process. 65, 904 (2017). https://doi.org/10.1109/TSP.2016.2616334, Google ScholarCrossref
  17. 17. T. Sauer, J. A. Yorke, and M. Casdagli, J. Stat. Phys. 65, 579 (1991). https://doi.org/10.1007/BF01053745, Google ScholarCrossref, ISI
  18. 18. M. Golubitsky and V. Guillemin, Stable Mappings and Their Singularities (Springer US, 1973). Google ScholarCrossref
  19. 19. C. K. Law, Combustion Physics (Cambridge University Press, 2006). Google ScholarCrossref
  20. 20. M. Kooshkbaghi, C. E. Frouzakis, K. Boulouchos, and I. V. Karlin, Combust. Flame 162, 3166 (2015). https://doi.org/10.1016/j.combustflame.2015.05.002, Google ScholarCrossref
  21. 21. M. Ó. Conaire, H. J. Curran, J. M. Simmie, W. J. Pitz, and C. K. Westbrook, Int. J. Chem. Kinet. 36, 603 (2004). https://doi.org/10.1002/kin.20036, Google ScholarCrossref
  22. 22. R. J. Kee, F. M. Rupley, E. Meeks, and J. A. Miller, “CHEMKIN-III: A FORTRAN chemical kinetics package for the analysis of gas-phase chemical and plasma kinetics,” Technical Report, Sandia National Laboratories, Livermore, CA, 1996. Google Scholar
  23. 23. T. Berry and J. Harlim, Appl. Comput. Harmon. Anal. 40, 68 (2016). https://doi.org/10.1016/j.acha.2015.01.001, Google ScholarCrossref, ISI
  24. 24. A. Singer, Appl. Comput. Harmon. Anal. 21, 128 (2006). https://doi.org/10.1016/j.acha.2006.03.004, Google ScholarCrossref
  25. 25. M. Budišić, R. Mohr, and I. Mezić, Chaos 22, 047510 (2012). https://doi.org/10.1063/1.4772195, Google ScholarScitation, ISI
  26. 26. M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, J. Nonlinear Sci. 25, 1307 (2015). https://doi.org/10.1007/s00332-015-9258-5, Google ScholarCrossref, ISI
  27. 27. E. M. Bollt, Q. Li, F. Dietrich, and I. Kevrekidis, SIAM J. Appl. Dyn. Syst. 17, 1925 (2018). https://doi.org/10.1137/17M116207X, Google ScholarCrossref
  28. 28. C. Bandt and B. Pompe, Phys. Rev. Lett. 88, 174102 (2002). https://doi.org/10.1103/PhysRevLett.88.174102, Google ScholarCrossref, ISI
  29. 29. R. Talmon, I. Cohen, S. Gannot, and R. R. Coifman, IEEE Signal Process. Mag. 30, 75 (2013). https://doi.org/10.1109/MSP.2013.2250353, Google ScholarCrossref
  30. 30. L. C. Evans and W. Gangbo, Differential Equations Methods for the Monge-Kantorevich Mass Transfer Problem (American Mathematical Society, 1999). Google Scholar
  31. 31. M. Belkin, Q. Que, Y. Wang, and X. Zhou, in Proceedings of the 25th Annual Conference on Learning Theory, Proceedings of Machine Learning Research, Vol. 23, edited by S. Mannor, N. Srebro, and R. C. Williamson (PMLR, Edinburgh, Scotland, 2012), pp. 36.1–36.26. Google Scholar
  32. 32. L. N. Wasserstein, Probl. Inform. Transmission 5, 47 (1969). Google Scholar
  33. 33. C. Villani, Optimal Transport (Springer, Berlin, 2009). Google ScholarCrossref
  34. 34. V. Baladi, Positive Transfer Operators and Decay of Correlation (World Scientific Publishing Co. Inc., 2000). Google ScholarCrossref
  35. 35. D. Ruelle, Thermodynamic Formalism (Cambridge University Press, 2012). Google Scholar
  36. 36. E. M. Bollt and N. Santitissadeekorn, Applied and Computational Measurable Dynamics (Society for Industrial and Applied Mathematics, 2013). Google ScholarCrossref
  37. 37. E. N. Gilbert and H. O. Pollak, SIAM J. Appl. Math. 16, 1 (1968). https://doi.org/10.1137/0116001, Google ScholarCrossref
  38. 38. Q. Xia, Commun. Contemp. Math. 05, 251 (2003). https://doi.org/10.1142/S021919970300094X, Google ScholarCrossref
  39. 39. A. Singer and R. R. Coifman, Appl. Comput. Harmon. Anal. 25, 226 (2008). https://doi.org/10.1016/j.acha.2007.11.001, Google ScholarCrossref
  40. 40. C. J. Dsilva, R. Talmon, C. W. Gear, R. R. Coifman, and I. G. Kevrekidis, SIAM J. Appl. Dyn. Sys. 15, 1327 (2016). https://doi.org/10.1137/151004896, Google ScholarCrossref
  41. 41. N. Courty, R. Flamary, D. Tuia, and A. Rakotomamonjy, IEEE Trans. Pattern Anal. Mach. Intell. 39, 1853 (2017). https://doi.org/10.1109/TPAMI.2016.2615921, Google ScholarCrossref
  42. 42. O. Yair, M. Ben-Chen, and R. Talmon, IEEE Trans. Signal Process. 67, 1797 (2019). https://doi.org/10.1109/TSP.2019.2894801, Google ScholarCrossref
  43. 43. G. Froyland and E. Kwok, J. Nonlinear Sci. (2017). Google Scholar
  44. 44. C. Moosmüller, F. Dietrich, and I. G. Kevrekidis, arXiv:1907.08260v2 (2019). Google Scholar
  45. 45. N. H. Packard, J. P. Crutchfield, J. D. Farmer, and R. S. Shaw, Phys. Rev. Lett. 45, 712 (1980). https://doi.org/10.1103/PhysRevLett.45.712, Google ScholarCrossref, ISI
  46. 46. D. Aeyels, SIAM J. Control Optim. 19, 595 (1981). https://doi.org/10.1137/0319037, Google ScholarCrossref
  47. 47. J. Stark, D. Broomhead, M. Davies, and J. Huke, Nonlinear Anal. Theory Methods Appl. 30, 5303 (1997). https://doi.org/10.1016/S0362-546X(96)00149-6, Google ScholarCrossref
  48. 48. J. Stark, J. Nonlinear Sci. 9, 255 (1999). https://doi.org/10.1007/s003329900072, Google ScholarCrossref
  49. 49. J. Stark, D. Broomhead, M. Davies, and J. Huke, J. Nonlinear Sci. 13, 519 (2003). https://doi.org/10.1007/s00332-003-0534-4, Google ScholarCrossref
  50. 50. H. Whitney, Ann. Math. 37, 645 (1936). https://doi.org/10.2307/1968482, Google ScholarCrossref
  1. © 2020 Author(s). Published under license by AIP Publishing.