Latin hypercubes and all that: How DOE works

Making design exploration software speak the language of engineers and not mathematicians has been a focus of development since the industry’s inception. Even so, our recent case study was typical in referencing the Latin hypercube design-of-experiments method, the radial basis function for generating a response surface model, the non-dominated sorting evolutionary algorithm to generate a Pareto front—all prompting this look into some of the quantitative methods that drive design space exploration.

DOE fundamentals recap—A designed experiment is a structured set of tests of a system or process. Integral to a designed experiment are response(s), factor(s) and a model.

  • A response is a measurable result—fuel mileage (automotive), deposition rate (semiconductor), reaction yield (chemical process).
  • A factor is any variable that the experimenter judges may affect a response of interest. Common factor types include continuous (may take any value on an interval; e.g., octane rating), categorical (having a discrete number of levels; e.g., a specific company or brand) and blocking (categorical, but not generally reproducible; e.g., automobile driver-to-driver variability).
  • A model is a mathematical surrogate for the system or process.
  • The experiment consists of exercising the model across some range of values assigned to the defined factors.

In deciding what values to use—more precisely, in deciding a strategy for choosing values—the goal is to achieve coverage of the design space that yields maximum information about its characteristics with least experimental effort, and with confidence that the set of points sampled gives a representative picture of the entire design space. Numerous sampling methods exist to do this: which to use depends on the nature of the problem being studied, and on the resources available—time, computational capacity, how much is already known about the problem.

In a helpful taxonomic discussion, Noesis Solutions observes that DOE methods can be classified into two categories: orthogonal designs and random designs. The orthogonality of a design means that the model parameters are statistically independent. It means that the factors in an experiment are uncorrelated and can be varied independently. Widely used methods are fractional- and full-factorial designs, central composite designs and Box-Behnken designs.

Source: Noesis Solutions

“A factorial design has some disadvantages: initially it is usually unclear which factor is important and which is not. Since the underlying function is deterministic, there is a possibility that some of the initial design points collapse and one or more of the time-consuming computer experiments become useless. This issue’s called the collapse problem. Most classic DOEs are only applicable to rectangular design regions. And the number of experiments increases exponentially with increasing number of levels.”

What of the other kind? Noesis: A random design means that the model parameter values for the experiments are assigned on the basis of a random process, which is another widely used DOE method. The most commonly used random DOE method is the so-called Latin Hypercube Design (LHD).

Source: Noesis Solutions

“The collapse problem does not occur with LHDs. This is because if one or more factors appear not to be important, every point in the design still provides some information regarding the influence of the other factors on the response. In this way, none of the time-consuming computer experiments will turn out to be useless.”

Drill-down on some principal DOE methods

[Click to enlarge]
Examples of (a) random sampling, (b) full factorial sampling, and (c) Latin hypercube sampling, for a simple case of 10 samples (samples for τ2 ~ U (6,10) and λ ~ N (0.4, 0.1) are shown). In random sampling, there are regions of the parameter space that are not sampled and other regions that are heavily sampled; in full factorial sampling, a random value is chosen in each interval for each parameter and every possible combination of parameter values is chosen; in Latin hypercube sampling, a value is chosen once and only once from every interval of every parameter (it is efficient and adequately samples the entire parameter space). Source: Hoare et al.,
Theoretical Biology and Medical Modelling, 2008.

  • Full factorial designs—The experiment is run on every possible combination of the factors being studied. The most conservative of all design types, yielding the highest-confidence results, but at the highest cost in experimental resources. Sample size is the product of the numbers of levels of the factors: a factorial experiment with a two-level factor, a three-level factor and a four-level factor requires 2 X 3 X 4 = 24 runs. Too expensive to run in many if not most cases.
  • Fractional factorial designs—Experiment consists of a subset (fraction) of the experiments that would have been run on the equivalent full factorial design. The subset is chosen to expose information about the most important features of the problem studied, using only a fraction of the experimental runs and resources of a full factorial design. Exploits the sparsity-of-effects principle that a system is usually dominated by main effects and low-order interactions, and thus only a few effects in a factorial experiment will be statistically significant.
  • Latin hypercube designs—Latin hypercube sampling is a statistical method for generating a sample of plausible collections of parameter values from a multidimensional distribution. In statistical sampling, a square grid containing sample positions is a Latin square if (and only if) there is only one sample in each row and each column. A Latin hypercube is the generalization of this concept to an arbitrary number of dimensions, whereby each sample is the only one in each axis-aligned hyperplane containing it. When sampling a function of N variables, the range of each variable is divided into M equally probable intervals. M sample points are then placed to satisfy the Latin hypercube requirements; this forces the number of divisions, M, to be equal for each variable. This sampling scheme does not require more samples for more dimensions (variables); this independence is one of the main advantages of this sampling scheme. Another advantage is that random samples can be taken one at a time, remembering which samples were taken so far.
  • Plackett-Burman designs—Used to identify the most important factors early in design exploration when complete knowledge about the system is often unavailable. An efficient screening method to identify the active factors in a design using as few experimental runs as possible.
  • Central composite designs—Experimental design useful in response surface methodology for building a second-order (quadratic) model for the response variable without needing to use a complete three-level factorial experiment. After the designed experiment is performed, linear regression is used, sometimes iteratively, to obtain results.
  • Box-Behnken designs—A type of response surface design that does not contain an embedded factorial or fractional factorial design. Box-Behnken designs have treatment combinations that are at the midpoints of the edges of the experimental space and require at least three continuous factors. These designs allow efficient estimation of the first- and second-order coefficients. Because Box-Behnken designs often have fewer design points, they can be less expensive to run than central composite designs with the same number of factors. However, because they do not have an embedded factorial design, they are not suited for sequential experiments.
  • Taguchi orthogonal arrays—Instead of having to test all possible combinations like the factorial design, the Taguchi method tests pairs of combinations. This allows for collection of the necessary data to determine which factors most affect product quality with a minimum amount of experimentation. The Taguchi method is best used when there is an intermediate number of variables (3 to 50) and few interactions between variables, and when only a few variables contribute significantly.
  • Taguchi robust design arrays—Taguchi robust design is used to find the appropriate control factor levels in a design or a process to make the system less sensitive to variations in uncontrollable noise factors—i.e., to make the system robust.

Today’s headline after the classic 1066 and All That

Following on this survey of design space exploration methods, a subsequent post will review design optimization techniques.

Design space exploration industry timeline

A timeline of company formations, product launches and M&A activity among design exploration and optimization software vendors maps the pace and direction of the industry’s development.

Click to view Continue reading

Model-based design optimization of a hybrid electric vehicle

Last week’s post surveyed the trend of integration between design exploration and optimization software and systems modeling and 0D/1D simulation tools. This week’s case study shows how the two technologies were used together in development of a new hybrid electric vehicle (HEV) to achieve the competing goals of improving fuel efficiency and meeting emissions targets.

Executive summary—A leading automotive OEM used Noesis SolutionsOptimus design optimization and process integration software in conjunction with Maplesoft’s MapleSim multi-domain systems modeling and simulation tool in developing the HEV’s combined electric and combustion propulsion system. Optimus was used to automate the traditional guess-and-correct simulation-based design process, its optimization algorithms efficiently directing the system simulation campaign to identify the best HEV configurations. Using Optimus’ capabilities for design of experiments (DOE), response surface modeling (RSM) and multi-objective optimization (MOO), engineers improved fuel efficiency by 21% and traveling performance (legal emissions compliance) by 15%. With MapleSim providing HEV modeling and Optimus controlling simulation workflow execution, the design optimization was accomplished in just two weeks. Continue reading

Model-based design exploration and optimization

Discussions of how to simulate early in product development fixate too often on FEA, overlooking the power of systems modeling and 0D/1D simulation for studying, exploring and optimizing designs at the beginning of projects, when product geometry is seldom available for 3D CAE but engineering decision-making can have its greatest impact and leverage on project success. Continue reading

Visual data analytics for design space exploration


Design exploration studies frequently produce large data sets for which it is useful to have tools for post-processing the data and presenting it in ways that facilitate visual discernment of patterns and information in the data. Visual data mining and analytics tools allow plots and tables to be viewed, queried and operated on to better understand the design space, explore design sensitivities, visualize correlations and investigate tradeoffs. Many design exploration software products include these capabilities to a greater or lesser degree, as do most mainstream CAE product lines. At the same time, some of the best regarded technology comes from developers focused exclusively on this area. Continue reading

Take our software user satisfaction survey

If you use design space exploration software, we invite you to take our five-minute survey of satisfaction with your software and vendor, and the benefits you’re realizing, via one of the links below. In return you’ll receive a report of the findings that will let you benchmark your experiences against those of your peers and competitors. Your participation and responses are strictly confidential. Findings will be discussed in aggregate only; no information about individual responses will be released. This survey is our own undertaking and is not commissioned by, nor executed in cooperation with, any software vendor or other party.

To take the survey, click the link for the software you use. If you use more than one brand, take the survey for each brand you use. If you don’t see your software here, click the Other brand link and write in your brand where the survey asks for it. We value your input and look forward to sharing the findings with you.

Innovation and new technology insertion


For engineering organizations, where does innovation come from?

When we put that question to EPC firms serving the process and power industries, the most frequent answer was “our projects” and the people working directly in project execution. Forty percent of respondents said their firms’ most important source of innovation is the discovery and application of new technologies and approaches by discipline leads, engineers and managers seeking solutions to pressures and exigencies in a specific project or program.

In second place was “anywhere and everywhere”—27% said innovation at base is a function of their organizations’ culture, and thus can arise from any area in the firm.

In third place was the IT department, named as the top source of innovation by 17% of respondents. While not quite the picture painted in some CIO-oriented publications, these findings align with what our research and others’ suggests is an evolving role for the CIO’s office: to provide enabling infrastructure in support of digital technology initiatives that, more and more, originate from the project execution centers of engineering, manufacturing and construction enterprises. Continue reading

Design space exploration: Justifying the investment

The foundational business value of design space exploration is the ability it confers on engineering teams and organizations to gain more complete, higher-fidelity visibility into product performance earlier in project schedules than was possible or practicable with older technologies and approaches. In essence, it does this by enabling more efficient, effective and revealing application of simulation, analysis and digital prototyping assets—tools, expertise, methods, work processes—to the perennial business drivers for any organization’s investments in those assets:

  • To become more competitive by gaining increased capability to explore, create and innovate.
  • To apply that capability to create better performing products.
  • To improve product quality and reliability—yielding expanded opportunity and customer appeal at the same time as lowered warranty expenses, liability exposure and lifecycle costs.
  • To control or, better yet, reduce product development schedules and budgets by supplanting costly, time-intensive physical testing with digital prototyping.

Continue reading

Design exploration software industry M&A outlook 2015

After 2013 saw Red Cedar Technology acquired by CD-adapco and FE-DESIGN by Dassault Systèmes, mergers and acquisitions in the design exploration and optimization software industry took a breather last year. What could drive M&A activity in 2015? Continue reading

Parametric vs. non-parametric optimization

Parametric shape optimization “searches the space spanned by the design variables to minimize or maximize some externally defined objective function” (Jiaqin Chen, Vadim Shapiro, Krishnan Suresh and Igor Tsukanov, Spatial Automation Laboratory, University of Wisconsin–Madison, “Parametric and Topological Control in Shape Optimization,” Proceedings of ASME 2006 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference). “In other words, parametric shape optimization is essentially a sizing problem that is a natural extension of parametric computer-aided design.

“The downside of parametric shapes is that they do not provide any explicit information about the geometry or topology of the shape’s boundaries. This, in turn, leads to at least two widely acknowledged difficulties: boundary evaluation may fail, and topological changes in the boundaries may invalidate boundary conditions or the solution procedure.”

Non-parametric optimization, by contrast, operates at the node/element level to derive an optimal structure. It can offer greater design freedom, and can make use of existing CAE models without the need for parameterization. “The main advantage of non-parametric shape optimization is the ease of setup, avoiding tedious parameterization that may be too restrictive with respect to design freedom” (Michael Böhm and Peter Clausen, FE-DESIGN GmbH, “Non-Parametric Shape Optimization in Industrial Context,” PICOF (Problèmes Inverses, Contrôle et Optimisation de Formes) ’12). “One of the major disadvantages on the other hand is that the CAD interpretation of the shape optimization result is not trivial.” Continue reading