Operation Research - Validation of a Model

Q1.What does the author say about whether a model can be completely validated?

A1. The validation of a model is the testing of similarities between behavior of model and the system being modeled. The question of how to measure the validity depends on the real world being analyzed and the typed of model being used. Linear Programming model of physical and engineering systems have physical validity that is results can be shown to work within real world environment. Simulation model of an ongoing physical system must be able to replicate the past to be acceptable as a decision aid. However, for the development of futuristic models, validity is superseded by the concept of credibility. Hence analysts do not believe that the concept can be completely validated and at best can be invalidated.

Q2. Summarize the distinctions made between model validity, data validity,logical/mathematical validity, predictive validity, operational validity and dynamic validity.

A2. The distinctions have been summarized below :-

Model Validity: Correspondence of the model to the real world and is concerned with identifying all the stated and implied assumptions, identification and inclusion of all decision variables and hypothesized relationships between variables. Assumptions are mathematical assumptions including the model form and continuity of the relationships ; content assumptions dealing with the scope and definition of model terms and variables; casual assumptions concerning assumed or hypothesized relationships between terms and variables. The analyst compares each assumption and hypothesis to the internal and external problem environments as viewed by the decision maker and make a statement regarding the extent of divergence.

Data Validity: It deals with raw data and structured data.Structured data are raw data upon which upon which some manipulation has been performed. Raw data validity is concerned with problems of measurement and determining if data are true in terms of accuracy (the ability to correctly identify, obtain and measure what is desired); impartiality – assurance that the data is recorded correctly; representativeness- assurance that the universe from which any sample data are drawn is properly identified and that the sample was random. Structured data requires review of each step of manipulation and is a part of model verification

Logical/Mathematical Validity is concerned with translating model form into numerical computer process that produces solutions. This involves aspects of model verification like determining if mathematical and numerical solutions are correct and accurate ; analyzing if the logical flow of data and intermediate results are correct, ensuring no omission of variables and relationships. There is no standard methodology for determining its logical validity.

Predictive Validity is the analysis of errors between actual outcomes and predicted outcomes for a model’s components and relationships. Outcomes can be parameters used as part of the internal computational process of the model, or an alternative solution selected by the model’s evaluative process. We look for the errors and their magnitudes, the reasons they exist, and if and how they can be corrected. It can be done by statistical tests, comparisons with historical data, predictive accuracy over time etc. Tests applied to each component produce information, the sum of which should enable the analyst to evaluate the technical validity of the model.

Operational Validity: This tries to assess the importance of the errors found under technical validity. Operational validity must conclude if the use of the model is appropriate for the observed and expected errors. The analysis should produce information that will enable the decision maker to conclude whether to accept or reject the model solution.As this is difficult, it requires close interaction between developers and decision maker to ensure that basic assumptions and trial outcomes are consistent with the decision makers expectations. The analyst must ensure that the costs are attributed correctly,the method for determining the benefits is valid, and stated benefits are attainable. Also checking is done if the model can produce unacceptable answers for proper ranges of parameter values.