NASA/TM–2016-219004 Decision Manifold Approximation for Physics-Based Simulations Jay Ming Wong University of Massachusetts Amherst, Amherst, Massachusetts Jamshid A. Samareh L angley Research Center, Hampton, Virginia January 2016 NASA STI Program . . . in Profile Since its founding, NASA has been dedicated to the  CONFERENCE PUBLICATION. advancement of aeronautics and space science. The Collected papers from scientific and technical NASA scientific and technical information (STI) conferences, symposia, seminars, or other program plays a key part in helping NASA maintain meetings sponsored or this important role. co-sponsored by NASA. The NASA STI program operates under the auspices  SPECIAL PUBLICATION. Scientific, of the Agency Chief Information Officer. It collects, technical, or historical information from NASA organizes, provides for archiving, and disseminates programs, projects, and missions, often NASA’s STI. The NASA STI program provides access concerned with subjects having substantial to the NTRS Registered and its public interface, the public interest. NASA Technical Reports Server, thus providing one of the largest collections of aeronautical and space  TECHNICAL TRANSLATION. science STI in the world. Results are published in both English-language translations of foreign non-NASA channels and by NASA in the NASA STI scientific and technical material pertinent to Report Series, which includes the following report NASA’s mission. types: Specialized services also include organizing  TECHNICAL PUBLICATION. Reports of and publishing research results, distributing completed research or a major significant phase of specialized research announcements and feeds, research that present the results of NASA providing information desk and personal search Programs and include extensive data or theoretical support, and enabling data exchange services. analysis. Includes compilations of significant scientific and technical data and information For more information about the NASA STI program, deemed to be of continuing reference value. see the following: NASA counter-part of peer-reviewed formal professional papers but has less stringent  Access the NASA STI program home page at limitations on manuscript length and extent of http://www.sti.nasa.gov graphic presentations.  E-mail your question to [email protected]  TECHNICAL MEMORANDUM. Scientific and technical findings that are  Phone the NASA STI Information Desk at preliminary or of specialized interest, 757-864-9658 e.g., quick release reports, working papers, and bibliographies that contain minimal  Write to: annotation. Does not contain extensive analysis. NASA STI Information Desk Mail Stop 148  CONTRACTOR REPORT. Scientific and NASA Langley Research Center technical findings by NASA-sponsored Hampton, VA 23681-2199 contractors and grantees. NASA/TM–2016-219004 Decision Manifold Approximation for Physics-Based Simulations Jay Ming Wong University of Massachusetts Amherst, Amherst, Massachusetts Jamshid A. Samareh Langley Research Center, Hampton, Virginia National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-2199 January 2016 Acknowledgments The This work was made possible by the NASA Internships, Fellowships, Scholarships (NIFS) program at NASA Langley Research Center. We thank Sriram Rallabhandi for FUN3D related help; Adam Weber for his help in computing the normal vector after trajectory convergence; Jing Pei and Jacqueline Clow for their support, interest, and help in this project; and Thomas Perry, Sherri Deshong, Richard Garfield, Rashinda Davis and Donna Mizell for their assistance throughout this project. We acknowledge the support of the Vehicle Analysis Branch of Systems Analysis and Concepts Directorate, the Big Data Analytics Group, and the Entry Systems Modeling technology development project. Lastly, we would like to thank Manjula Ambur and Ed McLarney for their continuous support of this activity. The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics and Space Administration. Available from: NASA STI Program / Mail Stop 148 NASA Langley Research Center Hampton, VA 23681-2199 Fax: 757-864-6500 Abstract With the recent surge of success in big-data driven deep learning problems, many of these frameworks focus on the notion of architecture design and uti- lizing massive databases. However, in some scenarios massive sets of data may be difficult, and in some cases infeasible, to acquire. In this paper we discuss a trajectory-based framework that quickly learns the underlying decision man- ifold of binary simulation classifications while judiciously selecting exploratory target states to minimize the number of required simulations. Furthermore, we draw particular attention to the simulation prediction application idealized to the case where failures in simulations can be predicted and avoided, provid- ing machine intelligence to novice analysts. We demonstrate this framework in various forms of simulations and discuss its efficacy. J. M. Wong & J. A. Samareh: Page 1 of 23 Contents 1 Introduction 4 2 Knowledge Bot 4 2.1 Definition and Mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 Extensions and Reworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2.1 Autonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2.2 Knowledge-Driven Selection of Target States . . . . . . . . . . . . . . . . . . . . . . . 5 2.2.3 Manifold Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 Formulation 5 3.1 Mathematics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1.1 Objective Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.1.2 Reward Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.2 Manifold Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2.1 Trajectory Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2.2 Gradient Normal Projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2.3 Obtaining Gradient and Normal Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2.4 From Trajectories to Manifold . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4 Demonstration 9 4.1 Isotropic Decision Boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1.2 Learned Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4.1.3 Parameter Influences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.2 Disjoint Isotropic Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.2.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.2.2 Learned Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.2.3 Parameter Influences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.3 Numerical Methods: Hyperbolic Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.3.1 Preliminary Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.3.2 Learned Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5 Simulations 14 5.1 Planetary Direct Entry Simulations with POST2 . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.1.1 Preliminary Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.1.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.1.3 Learned Classification Boundaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.1.4 Statistical Analysis of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 5.1.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 5.1.6 Influence of Discretized State Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2 Inviscid Flow of OM6 Wing Using FUN3D. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2.1 Preliminary Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2.2 Dataset Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.2.3 Decision Boundary and Physical Interpretation . . . . . . . . . . . . . . . . . . . . . . 18 5.2.4 Accuracy of Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.2.5 Larger Offline Simulation Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.2.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 6 Conclusions 20 References 22 J. M. Wong & J. A. Samareh: Page 2 of 23 List of Algorithms 1 TrajectoryOntoManifold: Learns trajectory T that converges to the unknown manifold 7 2 TargetFromNormals: Returns target t(cid:48) from unit normal space with maximal separation 9 3 ApproximateManifold: Learns a hyperplane approximation H that decides simulation S . 9 List of Figures 1 Proposed pipeline for our framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Gradient-based policy converges manifold (left), guaranteed due to the convex property (right) 7 3 Infinitely many normal vectors are present for any given estimated trajectory gradient . . . . 8 4 Reward function for isotropic boundary problem exhibits the convex property . . . . . . . . . 10 5 Support vector machine classification visualizations of isotropic decision manifold . . . . . . . 10 6 Accuracy vs. number of isotropic simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 7 Support vector machine classification visualizations of disjointed isotropic decision manifold . 12 8 Accuracy vs. number of disjointed isotropic simulations . . . . . . . . . . . . . . . . . . . . . 12 9 Classification results of the hyperbolic wave equation solver in previous work[1, 2]. . . . . . . 13 10 Support vector machine classification visualizations of the decision manifold for hyperbolic wave equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 11 Classification results in previous works for POST2 Venus direct entry . . . . . . . . . . . . . 15 12 Accuracy of manifold approximation framework on POST2 simulations . . . . . . . . . . . . . 16 13 Comparison between discretized state selection for POST2 simulations . . . . . . . . . . . . . 17 14 Decision boundary demonstration for inviscid flow on ONERA M6 (OM6) wing with FUN3D 18 15 Accuracy with coarsely discretized offline set for demonstration of FUN3D OM6 wing flow . . 19 16 FUN3D decision boundaries with finer grid and more simulations . . . . . . . . . . . . . . . . 19 17 Comparison between discretized state selection for FUN3D simulations . . . . . . . . . . . . . 20 J. M. Wong & J. A. Samareh: Page 3 of 23 1 Introduction Withtherecentsuccessofmachinelearningmethods, especiallywithlargedeepnetworks1, thefieldofcom- puter vision has undergone large advancements where learned features outperformed rule-based descriptors thathavebeenthestateoftheartforoveradecade[3–7]. Themethodofbuildinglargedeepneuralnetworks with massive training sets has soon propagated into other fields and applications as well, resulting in un- precedentedperformance[8–10]. However,inmanyofthesecases,thesuccessoftheselearningframeworksis largelydependentonhavingbigdatasets. Forinstance,manyoftheImagenetframeworkstrainingprocesses use large interconnected structures that rely on millions of human-labeled training examples [11–18]. In the case of the image classification problem, labeled training examples are abundant and the development of these approaches aims to leverage the massive collections of examples. Again, in the domain of other problemsasidefromimageclassification, largesetsoflabeledexamplesmaybeabundantoreasilyobtained, making techniques such as these extremely viable for learning. Forthecontextofthispaper,wepointtoauniqueapplication: thelearningofsimulationresults. Machine learning algorithms in the context of learning simulations discussed in various works have been shown to be both feasible and effective [1, 2, 19, 20]. Some of these approaches draw connections to image-related problems, building massive network architectures for learning when provided with large sets of training examples [1, 2]. The traditional big-data approaches require enormous training sets that in some cases may be infeasible to obtain, especially for complex simulations requiring immense computational power. The complexity—both in the underlying simulation and large input dimensionality—makes naive learning methods infeasible. In this paper, we propose a reformulation of a deep learning framework that is suitable for complex simulations requiring long execution time. Rather than relying on a massive collection of training examples, we investigate the notion of intelligent selection of target parameters. The approach consists of a reinforce- ment learning approach that quickly drives states along some learned trajectory towards the surface of the decisionmanifold. Afterwards,asupportvectormachinewithradialbasiskernelexpansionisusedforlinear separation in infinite dimensional space. 2 Knowledge Bot 2.1 Definition and Mechanics Theconceptoftheknowledge bot isfirstintroducedasanautonomousclassifierwhichlearnsbycontinuously samplingaspecificdomainandrefiningitsunderlyingresultingmodelfromaparticularsimulation[1,2]. An analystwhowishestoperformasimulationprovidesinputparameterstothesimulationforexecution. Rather than executing the simulation, the knowledge bot intercepts these parameters and provides fast predictions regarding the success of the simulation. The prediction was shown to be over 99% accurate in the previous works[1, 2]. In essence, the knowledge bot serves as an abstraction of the simulation software providing accurate feedback as to whether the actual execution will result in success or failure, avoiding simulating problemsthatarepredictedtoresultinfailure. Thebotencapsulatestheexpertiseofatrainedanalystwith many years of experience and is familiar with parameter settings that result in failure through learning the underlying simulation idiosyncrasies. There are two major motivations for developing the knowledge bot. First,atrainedanalysthaslearnedtheabilitytodetectfailingparametersforsimulationtools. Afteryearsof experience,theyarequitegoodathavingtheinsighttoknowwhatinputparameterswillcausethesimulation toeitherfailorcrash. Theknowledgebotaimstoencapsulatethelearnedexpertiseofatrainedanalystintoa predictivetoolthatanoviceanalystmaybeabletoquerywhetherornotsimulationparametersarevalidfor simulation. Second, instead of having analyst simulation parameters that are predicted to fail (even trained analysts sometimes run into this issue), the knowledge bot may be able to predict the simulation failure quickly, saving computational resources and analysis time. Moreover, these two motivations are connected by a single theme of allowing for rapid development and analysis without wasting time and resources on failed simulations. This is particularly important for the design of aerospace vehicles. 1Preliminaryknowledgeinthispaperassumesourpreviousworkinbuildingknowledgebots,orsimulationclassifierstrained vialargenetworkstructures[1,2]. J. M. Wong & J. A. Samareh: Page 4 of 23 2.2 Extensions and Reworks Intheearlierworks[1,2],theknowledgebotlearnedfromthecompleteinputtooutputmappingofsimulation parameters. However in practice, we are only concerned with states close to the boundary of the separation between success and failure. Because the problem is of a binary separation nature, states can easily be classified according to its relative position with respect to the decision boundary, or in the case of muti- dimensions, the decision manifold. In this paper, we introduce extensions to the previously introduced concept of knowledge bots[1, 2] while maintaining a sense of abstraction from the individual application. The three major concepts we discuss are as: 1) autonomy, 2) knowledge-driven selection of target states, and 3) manifold approximation. 2.2.1 Autonomy –Theautonomousbehavioroftheknowledgebotisachievedbyconsideringsomepolicy P that relates the tendency for exploration and exploitation of the candidate targets in the state space. Furthermore, it is important to produce solutions at any time that are refined as more time is allocated to learning. In other words, the knowledge bot must be able to operate with minimal simulation results while producing sufficiently accurate predictions. In addition, the bot must autonomously discover new states under some policy P that can be used to refine its learned model. 2.2.2 Knowledge-Driven Selection of Target States – In previous works[1, 2], the knowledge bot relied on pseudo-random numbers to determine candidate states to evaluate; although this approach results in a learned model that eventually converges to a model that predicts the outcomes with unity accuracy, it requires infinite exhaustion through all possible configuration of values in all possible state discretization schemes. 2.2.3 Manifold Approximation –Insteadofattemptingtolearnthemappingof(possiblyinfinite)pairs of simulation input parameters to simulation termination results, in the proposed framework we consider onlylearningthedecidingregionsofthestatespace. Inotherwords,thiscorrespondstothemostinteresting states along the decision manifold, which indicates the decision boundary between states are that extremely volatiletosmallperturbationtotheirconfigurationvalues. Inthesenseoftheframework,thesetargetstates correspond to or have a high probability in being support vectors for the supervised support vector machine method to approximate the decision manifold. 3 Formulation The authors have proposed a framework to capture analyst expertise in a knowledge bot by performing thousands of simulations as training examples for an artificial neural network[1, 2]. However, in some cases, acquiringthousandsofsimulationresultsmaynotbefeasibleforcomplexsimulations. Anintelligentmethod in determining simulation success or failure is necessary when these problems emerge. It is important to know that success and failure simulation results are highly structured. Simply speaking, generally small perturbations in the input parameters for a successful simulations will still tend to lead to success. This is not the case when the parameters approach the manifold of success and failure. In practice, learning the decision manifold that separates the two classes is sufficient. This realization sparked our proposed framework that focuses on quickly identifying potential states (or configurations) close to the decision manifold using reinforcement learning to drive the state of the systemsufficientlyclosetothemanifold’ssurfaceformingalearnedoptimaltrajectorythatconvergestothe decisionsurface, andapproximatingthemanifoldthroughsupportvectormachinewithradialbasisfunction (an infinite-dimensional kernel expansion). ThebasicpipelinefortheproposedframeworkisillustratedinFigure1. Thepipelineconsistsofaniterative stepoftrajectoriesofstatesc ∈C ord ∈D(ConvergentandDivergent)thatconvergetotheboundary i j of the manifold. The states close to the surface of the manifold can be thought of as support vectors that define the manifold itself. The states are extracted from the resulting trajectories and are used to train a support vector machine classifier with a radial basis function kernel approximating the true underlying decision manifold. J. M. Wong & J. A. Samareh: Page 5 of 23 Find converse state w.r.t target value Determine initial Learn trajectory to Evaluate target Support Vector Trajectories target states manifold (Simulation) Machine (RBF) Discover new target by gradient normal Figure 1. Proposed pipeline for our framework 3.1 Mathematics 3.1.1 Objective Function – The discovery of an unknown decision manifold of success and failure of some simulation can be thought of as an optimization problem where we define two sets C and D, corre- spondingtotheConvergentandDivergentsimulationresultsgivenaparticularsetofinputparameters x ,x ,··· ,x ∈ X. In the learning framework we can consider these input parameters as configuration 1 2 n variables q ,q ,··· ,q that define the state of the system with an output observation of y ∈Y. The output 1 2 n y determines whether the state of the system belongs to a convergent C or divergent D set. The decision manifold lies between the two sets. In other words, a particular configuration is near the decision manifold when it is a solution to the following objective function for a particular trajectory, argmin ||c −d || for i,j =1,2,··· ,k (1) i j 2 q1,q2,···,qn Note that c ∈ C and d ∈ D consist of states along a particular trajectory. If we can drive the state to i j some solution of the above objective function following some trajectory, then that state is sufficiently near the decision manifold. The overall optimization problem can be framed as follows, minimize ||c −d || for i,j =1,2,··· ,k i j 2 q1,q2,···,qn subject to Q− ≤q ≤Q+ 1 1 1 Q− ≤q ≤Q+ (2) 2 2 2 . . . Q− ≤q ≤Q+ n n n where Q is the set of configuration domains, and therefore Q− and Q+ correspond to the lower and upper i i bounds of the ith configuration variable. The set of configuration variables q ,q ,··· ,q defines some 1 2 n configuration or state. 3.1.2 Reward Function – A canonical reward function for any particular state or target configuration t=(q ,q ,··· ,q ) can be written as, 1 2 n (cid:40) −min||t−d || for j =1,2,··· ,|D| if t∈C R (t)= j 2 (3) t −min||t−c || for i=1,2,··· ,|C| otherwise i 2 Wemodifytheobjectivefunctionsuchthattherewardismonotonicallyincreasingastargetconfigurationsare close to the decision manifold. The reward value for a particular target configuration is simply computed as theEuclideandistancebetweenitandtheclosestobservableconversestate. Weseethatthisrewardfunction satisfies an underlying convex potential property where the reward value for states grow monotonically towards the optima which lies sufficiently close to the decision manifold. J. M. Wong & J. A. Samareh: Page 6 of 23