# Challenges to CMM precision ... and the rewards new software solutions will bring to mating parts.

Challenges to CMM precision

It's the rage - the coordinate measuring machine. Every shop has to have one. But simply having one and enjoying its benefits are two different stories. New users are discovering something the veteran CMM users have known for years: it takes highly trained people to fully understand and properly use this exceedingly complex machine.

Measurement has always been critical to the manufacturing arena, but with a CMM, the measurement math you use - its software algorithms - is also critical to the results you get. At the moment, these algorithms, as well as the standards used to evaluate them, are embroiled in controversy. Until it's resolved, it will surely affect users' confidence in the precision of their measurements - particularly mating surfaces.

Meanwhile, although CMM vendors and users alike may claim their CMMs are being used in full compliance with measurement standards ASME Y14.5 and B89.1.12, this is an overstatement. There is no comprehensive CMM gaging standard. Nevertheless, once these issues are resolved, the benefits to manufacturing will be enormous.

The challenge for dimensional metrology is to measure the real world, where nothing is perfectly flat, round, or square. All manufactured parts vary in size, feature location, and form. Further, all measuring techniques, in and of themselves, add error.

Whether you use a surface plate or CMM, you can only measure these imperfect parts with a degree of uncertainty that requires definition: metrologists must maintain the correct proportion between the uncertainty in their measurement techniques and the tolerances of the measured part.

Why Y14.5?

Geometric dimensioning and tolerancing (GD&T) was born in the 1950s to help stamp out measurement ambiguity at its source - when drawings are made and tolerances set. The ASME Y14.5M.1982 tolerancing standard for GD&T relates all dimensions to a set of mutually perpendicular planes, called a datum reference frame. A datum is a theoretically exact point, line, or plane derived from a measured part's physical surface. A datum feature is simulated by the associated machining table or surface plate. In GD&T, all feature measurements are related to specific, prioritized datum planes, with the part in a specific orientation, so that a part is always measured in exactly the same way; i.e., independent measurements should yield the same results. By being definitive about fit and function, GD&T can save time, save materials, reduce rework, and eliminate measurement disputes.

Unfortunately, Y14.5's complexity has inhibited its effective use - it's widely used, but not widely understood. Complexity isn't its only problem. Since the standard was last modified in 1982, machining technology has caught up to gaging technology - the historical 10:1 accuracy edge between gage and part no longer exists. Further, widespread CMM use has exposed the fact that Y14.5 was written for hard surface gaging, not "soft" mathematical surfaces that CMMs use to represent planes and part surfaces.

The present Y14.5 standard assumes that the part will be measured on a "perfectly flat" surface plate. Any inadvertent mismatch in the waviness of the part and plate surfaces can create error. When you're measuring parts in the 0.0003" to 0.0005" range, for example, the surface plate becomes part of the problem.

With the factor of ten replaced by a factor of two or three, matching the error distribution from your process with that for your measuring device becomes a problem. Before, you could ignore this statistical distribution. Today, you cannot.

Invisible planes

To establish a reference plane, many CMMs use a theoretical "average" plane that does not exist on the part's surface. Using a set of data points and an algorithm called "least squares(*)," they draw an imaginary plane that would actually lie somewhere under the contact or tangent plane of that part if it were lying on a surface plate. Y14.5 specifies that a tangent plane (a plane touching the highest points on the part surface) be used as a reference plane to simulate the surface upon which that part mates. But where are the three highest points to establish that contact plane? The CMM can only guess at it statistically. Resolving this incongruity is a primary goal of the Y14.5.1 "Mathematics of Geometric Dimensioning and Tolerancing" committee.

Y14.5 is commonly misperceived to be a gaging standard. It isn't. It is strictly limited to defining GD&T tolerancing. Yet, many of the examples used in the standard imply specific gaging techniques. Because Y14.5 needs to be updated to mathematically define points, surfaces, and forms, it will need to be completed before the upgrading of related measurement standards, such as B89.3.2, Methods of Inspection. These works are proceeding in parallel with the B89.3.2. work. Because the latter is several orders of magnitude more complicated than the Y14.5.1 task, it will take considerably longer.

The various B89 standards are not just concerned with CMMs, but with all gaging. (For example, the ASME B89.1.12 standard that helps define volumetric repeatability and linear accuracy for different CMMs does not define how to use the machine to make measurements.)

Y14.5 defines tolerance zones. For example, for flatness, Y14.5 states that all points on the surface should lie between a pair of parallel planes separated by the tolerance. For a 0.0001" tolerance, you need only prove that all points on the surface lie between some arbitrarily oriented set of parallel planes 0.0001" apart to show that the part is in compliance with the design intent. Whether you use a CMM's least-squares approximation, an optical flat, or an indicator on a surface plate is your option, but because you will be making assumptions and taking some risks, it is up to someone else to confirm that your measurement methods are realistic and acceptable.

A key event that stimulated the "mathematizing" effort was a GIDEP (Government/Industry Data Exchange Program) Alert issued in August 1988 by Richard Walker, a senior quality engineer for Westinghouse's Marine Div, Sunnyvale, CA. Since the mid '80s, he has been a member of the B89.1.12 software subcommittee working on coordinating the use of CMM software and verifying parts according to Y14.5. "We had become aware," he recalls," of a disparity between CMM software and the Y14.5 standard, and this prompted a meeting that included Y14.5 members, some B89.1.12 members, and others in the measurement community. Early on, we concluded that our biggest problem - overshadowing any software problems at that time - was the lack of knowledge of GD&T. Shortly thereafter, I was named chairman of a committee formed to address this problem on the B89.1.12 committee. We attempted to find a common ground to solve the problem of the disparity between sample-measurement systems and dimensions, and tolerances specified to the Y14.5 standard.

"Then, in 1988, a Westinghouse design engineer called to say that he was getting a CMM reading that a certain part's parallelism was better than its flatness (same points, same surface). That isn't possible!

"When I began to investigate this problem, I didn't realize that a disparity existed in the software's ability to take surface measurements and differentiate between parallelism and flatness. So, I created a test, and through it discovered an error in the parallelism algorithm of one vendor's software. Next, I created a data set and input that into computers with various CMM software programs. Different CMM software gave widely varying answers for the parameters of parallelism, perpendicularity, etc."

The bottom line was that Walker had shown that some CMMs were capable of passing bad parts with a measurement error as large as 37% and rejecting good parts with errors up to 50%. Admittedly, this was for the case of a test part with deliberately large surface error (1"). "Although I used some errors of a magnitude far removed from reality, I also brought that down to realistic errors of 0.001" and 0.0001". I simply used both ends of the scale."

(Please note that all of Walker's comments here are his own opinions, and not necessarily those of ASME, Westinghouse, or the committee he chairs.)

When he presented his findings to the B89.1.12 committee, which included representatives from all the major CMM manufacturers, they were unimpressed. His experiments seemed extreme, designed to insure discrepancy. Nonetheless, he and his employer decided to go ahead and issue the GIDEP Alert to share his findings with other CMM-using government contractors.

It created quite a stir. More than 500 CMM users called Walker for details on his tests and advice on how to improve the validity of their own measurements. This reaction proved to be a major stimulus to the Y14 and B89 standards work in progress. It also got the attention of the CMM manufacturers, whose response, Walker says, ranged from admission of software flaws to denials of any serious problems.

Roundness vs flatness

Obviously, when different CMMs yield different results from the same set of data input, users face a disheartening dilemma about which answers (and algorithms) are correct. Continues Walker, "Software differences among many CMMs still exist. For roundness, for example, almost all still use least-squares algorithms, when the B89.3.1 roundness standard states that minimum radial separation is the proper algorithm to use unless otherwise specified on the drawing. A CMM will give a totally different answer than a roundness-measurement machine. How great that difference is depends on both the magnitude of the errors in the part and differences in sampling density."

Although roundness has a definitive standard, flatness does not. "When flatness is specified on a drawing," says Walker," no flatness or parallelism standard (or datum standard) is available to guide the interpretation of that flatness and what is actually to be measured.

"Flatness is intuitively interpretable - easily defined mathematically - but datum structures present another problem. There is simply no mathematical rigor for interpreting a datum structure or contact plane. Therefore, using a least-squares plane as an estimate of the measurement can lead to unpredictable results."

How soon will we see results?

Walker is optimistic about his committee's ability to solve the mathematizing task. Although others on the committee might disagree, he feels they are already half finished. "I would say it will be ready for public review next year, and on the street in two years. Parts of the document were presented to the Y14 committee this spring. Other standards committees may argue over dots and dashes, or suffer from lack of corporate support; but we're lucky to have good, motivated people who want to get this work finished and in use."

"A number of issues need to be resolved," he explains." Certainly, we must develop software testing to the level where we know that all algorithms are giving the same answers. Here's what often happens: A vendor makes a part on his five-axis machine tool, measures it on his Brand A CMM, and ships it to us. We then measure it on our Brand B CMM, and get different answers for the same features. So, we call in that vendor and argue about what the drawing called for, who did what, who's at fault, etc.

"This kind of wasted effort represents a tremendous cost to US manufacturing, and the number of times this happens is phenomenal! It's a problem not just with CMMs, but with measurement in general. Whether you use a surface plate, a roundness gage, a shadow graph, etc - the bottom line is that industry doesn't know how to properly measure a part. The literature and training just aren't there. Some companies out of necessity have developed relationships with vendors to avoid this problem, but they have to go through a whole new learning curve if they change vendors."

Part of the problem is that each CMM manufacturer has its own proprietary software solution. "The GIDEP Alert just touched the tip of the iceberg. Mathematizing the datum reference plane is a much bigger problem. At the moment, there is no straightforward solution, except to incorporate the forthcoming definition from Y14.5.1."

In the interim, Walker suggests working with the CMM manufacturer's software, and doing some thorough testing to find what its shortcomings really are. "The CMM manufacturers are contributing to solving these problems," he reports. "They are aware of many of these measurement shortcomings and the proper corrective measurement techniques."

Form-fit error

Walker laments that CAD systems cannot interpret the proper manufacturing response for a given part's tolerance; i.e., what machine to produce it on, what machine to measure it on, and how to take those measurements. "Computer systems can't do that yet, so people must make those decisions after the design and tolerances have been locked in."

Walker points out that mathematizing the current GD&T standard will benefit manufacturing in general, and is not being done exclusively to solve the CMM problem. It will not give CMM manufacturers the ability to write proper algorithms for a given measured surface.

The closer a part approximates true form, the more accurate the measurement will be. This is because the software algorithm's job of fitting a chosen elemental shape to the actual part data is made mathematically easier, and thus its iterative calculations are faster and more accurate. A machined part with a well-defined form is much easier to measure than an irregular form, such as a casting.

"Using an algorithm to fit a surface to its sample-measurement points," Walker explains, "is more involved with the manufacturing process than with what the drawing calls for. The different undulations and perturbations in a surface produced by milling, grinding, EDM, etc require different algorithms to fit those surfaces. The key research that needs to be done concerns how surfaces are produced and what characteristics result."

Fortunately, Professor Robert Hocken at the University of North Carolina at Charlotte is doing that research. His work will help CMM users interpret measurements from sample data points, whereas mathematizing of Y14.5, Walker explains, will greatly improve communication between design and manufacturing.

NIST's software testbed

The work on improving and standardizing CMM algorithms will also be energized more by Hocken's studies than by mathematizing Y14.5, and that work has already begun at the National Institute for Science & Technology (NIST). With Hocken's help and initial software, NIST is creating a testbed to evaluate CMM algorithms and determine whether they produce the same answers for the same data input. The correct answers, however, won't be defined until the mathematizing is finished and the B89.3.2 activity on proper measurement techniques is completed. First peek at the NIST-tested algorithms was scheduled for October.

Using the NIST algorithm-testing system as the standard, a vendor will be able to say, "We have tested our CMM software with the NIST Testing System, and our roundness algorithm comes within 98%, our flatness algorithm comes within 97%, etc." This will be a public-domain system, so vendors will probably want to build onto it a proprietary edge in speed, interfacing, or special functions.

Long-range benefits

Walker feels that vendors should be able to modify their software soon after the mathematizing standard and Hocken's processing database are finalized. He feels CAD systems will be affected most. "It will allow them to understand the perfect geometry of a part, as well as its allowable variability when the tolerance is first established."

With a database of manufacturing capabilities at his command, a designer could ask "Which machine should I use to produce this part with this tolerance?" And it will reply, "Your Number 3 mill can produce 50 parts in 5 hr." Or maybe, "For that tolerance, you will need to purchase a machine with this specification, and it costs \$450,000." With this kind of instant feedback, designers may want to rethink tight tolerances.

"The cost savings that would result from having such a system would be phenomenal!" exclaims Walker. "Designers often don't understand the ramifications of the tolerances they set."

"In metrology," he continues, "the B89.3.2. activity will have its greatest effect on the commonality of measurements, making measurement definitive. When I want to check something for flatness, I will be able to get this response from the database: `Choose a CMM and take this type of sample data, and your measurement error will be within this range. Or, choose an indicator and surface plate of this variety, use this technique, and your measurement error will be within this range."

The sampling problem

In addition to mathematizing a reference plane, there is the issue of surface sampling: how many points must be measured to define a surface? "Three" is the all-too-common answer, when the statistically correct answer is something greater than that. Warns Walker, "No matter how many samples you take, some algorithms will still give you erroneous answers. Sampling needs a lot of research, and new probing systems must be evaluated. Because computers can now handle much more data than before, the latest CMMs don't have the same limitations that earlier ones did. Nevertheless, once you've purchased a machine as complex as a CMM, it's your responsibility to become knowledgeable in its use.

"At this point, the effectiveness of CMMs is truly limited by user's lack of knowledge. I feel that the greatest value of the CMM is in process control rather than in part verification after the fact. Don't use your machine simply to verify whether a part is good or bad. Learn to use it to improve your process."

Mating and other problems

Is there a significant mating problem today? Are parts not fitting together because we can't measure mating planes? "Yes," Walker replies. "I would say it's significant, and just as much so with a surface plate as with a CMM. Mathematizing Y14.5 will at least partially address the problem of the contacting plane and datum structure, but we can distribute this problem, and the need for improvement, in different areas. Roundness errors lie in CMM software, because that standard algorithm has been set. Flatness errors are a question of data-point density, and probably half the datum-structure problem lies in the need for a mathematical definition of Y14.5. The tolerancing standard is way behind industry right now. For example, there is no standard for solids-modeling tolerances - one of the significant areas we're working to define with mathematization."

Probing the future

A basic question for those currently committing to CMMs is whether the basic mechanics of touch-probe CMMs will prove too slow to define a contact plane. Although the basic touch probe will never measure millionths, some scanning-type probes may, and some of the advanced measurement probes certainly will.

Richard Walker sees promise in combining the touch probe and laser. "That combination has a lot to be said for it. The laser is not as accurate as the touch probe, so you use it to get a picture of the lay of the surface; then you use the touch probe to take your extreme points."

The form-error database

Dr Robert Hocken, one of the chief authors of the first B89.1.12 standard in 1985, former head of precision engineering at NIST, and current head of precision engineering at the University of North Carolina at Charlotte, has been heavily involved in standards work for years. His NSF-supported project "Sampling Techniques For Coordinate Measurement Machines" is an ambitious effort to develop a database of the expected form errors associated with all basic manufacturing processes, based on inputs from industries here and abroad.

"Then," he explains, "we will test sets of algorithms that we have developed for those basic features against systematic errors that actually occur, both in the process that makes the parts and in the measuring machines with their own systematic errors. We will then use different sampling techniques to find the best results.

"Because there are so many variables," he continues, "we will use a design-of-experiments approach. For instance, on a simple circle, we're using hundreds of data sets of different form errors, different waviness for the multiple lobes you can get with various processes. Centerless grinding usually gives you three lobes, but turning on a four-jaw chuck gives you four, and some types of cylindrical grinding on centers produce multiple lobes, depending on the resonances that occur. Although we're not attempting to make this all inclusive, we hope to come up with a set of guidelines that give you the best confidence you could expect, given certain sets of conditions."

As Hocken envisions it, such an expert system would be able to tell the user exactly how to measure a given feature machined on a given type of machine, and even show you where to take sample points. The software would then run a statistical check to make sure that the divergences (residual errors) are within bounds and the machining process has not changed, or it might suggest that you retake the measurement differently.

So far, Hocken has acquired considerable data and his survey is ongoing. He predicts the first database will be in usable form within a year, with continual expansion thereafter. On the issue of sampling points, Hocken hopes to be able to not only tell CMM users how many sample points they need for a given process, but also what kind of sampling scheme to use.

For example, in holemaking, he points out, "It isn't acceptable to take just three hits at 120 deg on three-lobed parts. Users should take at least seven points and use a stratified sampling technique or risk missing a form error for that shape and get a totally wrong answer."

Future of CMMS?

Will the new mathematization standard push CMM technology? "I'm sure it will," Hocken says. "My guess is that people will need to sample more points than they did before to get the confidence levels they thought they were getting. If so, the speed and probing capability of CMMs will have to increase, or they will be replaced by another technology. The standard's not going to lock in CMMs. If an optical probe or a fringe-counting system could take high-density coordinate data over a surface very fast, the standard would certainly permit that."

Like Walker, Dr Hocken feels the next generation of CMMs will use three-dimensional scanning touch probes for the higher accuracy applications and optical noncontacting methods for lower accuracies.

He predicts major cost benefits from his work to characterize manufacturing processes. "Interpreting how two parts function and how to properly put GD&T symbols on the drawing is the designer's problem and has the greatest cost ramifications. If the design process is done thoroughly and manufacturing processes are under process control and taken into account, you can eliminate the verification process. Like Walker, I have always felt that the proper use for the CMM is to develop process-control parameters, rather than simply verifying if parts are good or bad." (*) Least squares - the most commonly used CMM measurement algorithm - sums the squares of the deviations of each data point from a surface or curve drawn through the data set. That surface or curve is moved about (mathematically) until the sum of the squares is minimized. If the measurement data are normally distributed, least squares yields the best approximation of any algorithm. Unfortunately (particularly with small samples), "normal" distribution is often not the case. The least-squares fit is an approximation of the (average) surface, but the average surface may not be the best surface for evaluating the functionality of the part.

Editor's note: We would like to know what you think about these issues in CMM metrology. Is this a problem at your plant? Are you rejecting a vendor's CMM-approved parts? Are you using your CMM software properly? Are your quality people even aware that these problems exist? We invite your comments.