Do Resistors Really Hold 5% Tolerance?

Intro: Do Resistors Really Hold 5% Tolerance?

I just bought a bunch of 560 Ohm gold band resistors from a local surplus store. They were a nickel each. Is it possible to make millions of parts, sell them so cheap, and maintain the +/- 5% tolerance specified by the gold band? The tool used to answer this question is Statistical Process Control (SPC).

You will need:

A digital multimeter

A bunch of resistors the same value. (I measured thirty)

The spreadsheet template located in step five.

STEP 1: Statististical Process Control

To understand SPC it is necessary to understand the following terms:

Nominal is the exact measurement you are trying to achieve. But in reality nothing is ever perfect. There is variation in all processes, the Tolerance sets the limits of how much variation is acceptable. The Lower Specification Limit (LSL) and Upper Specification Limit (USL) are the limits of the tolerance.

The Mean is the arithmetic average of a set of values, or distribution. The Median is the point where half of the values are less and half are more. The Mode is the most common value. In an ideal situation the Nominal, Mean, Median, and the Mode will all be the same.You can get a rough estimate of how consistent your process is running by comparing them.

Process Capability (CP) is the measurement to determine if the process is capable of holding the tolerance allowed. To find the CP first you find the Standard Deviation. The Mean + ( Standard Deviation * 3) gives you the Upper Control Limit (UCL). Next find the Lower Control Limit (LCL), Mean - ( Standard Deviation * 3). The capability is the ratio of the specification limits over the control limits, (CP = (USL - LSL) / (UCL - LCL). If the CP equals one the control limits fit exactly within the specification limits.

You want it to be larger than one to give you some room for error. The reason for using +/- three standard deviations is because in a normal distribution 68.2% of the values will fall within 1 standard deviation. 95.5% will fall within 2 standard deviations, and 99.7% will fall within three. These figures are mathematical constants known as the Empirical Rule. As the amount of variation increases the standard deviation will also increase.

CPK is the measurement of how well centered the Mean is to the Nominal, if they are identical the CPK will equal the CP. More variation between the two in either direction will result in a lower CPK.

The standard most widely used in industry is a CPK of 1.3.

The math mentioned here gets complicated, but it is easy to estimate. With a normal distribution over the center half of the tolerance with the mean centered on the nominal your CPK will be approximately 1.3. An even distribution over the center half of the tolerance with the mean centered on the nominal will give you a CPK of approximately 1.1. If the CP and CPK both equal exactly one 99.7% of the parts will be within the tolerance. The other .3% will be bad.

STEP 2: Capable and in Control

In this illustration both of the control limits (LCL and UCL, plus and minus three times the standard deviation) are within the specification limits (LSL and USL) meaning the process is capable.

The vertical line represents the nominal specification and the top of the curve represents the process mean.

The mean is well centered on the nominal so the process is said to be in control.

CP and CPK are equal.

STEP 3: In Control But Not Capable

Here the control limits fall outside the specification limits, but the mean is well centered on the nominal.

The process is in control but not capable.

There is to much variation in the process. it must be improved so the control limits fall within the specification limits.

Again CP and CPK are equal.

STEP 4: Capable But Out of Control

In this illustration the control limits would fall within the specification limits if the mean was centered on the nominal.

The process is capable but out of control.

Make an adjustment to bring the mean in line with the nominal.

Here the CPK is lower than the CP.

STEP 5: Enter the Data and Get the Results

Download the two spreadsheets at the bottom of the step. 560Ohm.ods is my data, the same as shown here. SPC.ods is a blank SPC template you can use for this or any other SPC study.

Statistical Process Control
Capability Study Worksheet Company AxMan
Order # 560 Ohm +/-5% resistor USL - 588
LSL - 532 Sample
1 - 557
2 - 557
3 - 560
4 - 555
5 - 555
6 - 559
7 - 560
8 - 558
9 - 558
10 - 560
11 - 560
12 - 554
13 - 560
14 - 558
15 - 556
16 - 560
17 - 558
18 - 556
19 - 560
20 - 564
21 - 555
22 - 561
23 - 558
24 - 555
25 - 559
26 - 558
27 - 558
28 - 556
29 - 557
30 - 557 Nominal - 560.0000
Mean - 557.9667
Median - 558.0000
Mode - 558.0000 CP - 4.2036
CPK - 3.8983 LSL - 532.0000
LCL - 551.3057 UCL - 564.6277
USL - 588.0000 High Sample - 564.0000
Low Sample - 554.0000

The CP and the CPK are both a long way above 1.3, these resistors are within the specified tolerance with plenty of room to spare.

The resistors were purchased at Axman, a surplus store in St.Paul MN.
I have no financial interest in Axman, I am just a customer.

Comments

[baudeagle]

If you are interested in pursuing this further, you could conduct a gauge repeatability and reproducibility study. This would allow you to analyze how much error is attributable to the operator and the to the test equipment.

[JRV31]

For more information on statistical process control and capability
studies on more electronic components please check out my SPC
collection:

https://www.instructables.com/id/Statistical-Process-Control-3/

[APteH]

Is your meter calibrated?

[JRV31]

After much searching I finally found a 560 Ohm 1% tolerance resistor. It's on order now and when it arrives I can check my meter to a higher standard. I will post the results here.

[JRV31]

My 1% resistor arrived yesterday. It measured 559 Ohms on my meter.

[Omnivent]

Hi,

If you really want to test your meters calibration, at least 3 resistor values are needed - one low, one medium and one high, to check the precision over a range.

On say a 200k FSD range, suitable values could be e.g. 20k, 60k and 180k (just use a value near the 60k). Then you can calculate the deviation by inter- and extrapolation.

1% resistors are not gonna cut it, if you want it to be as exact as possible, but you may wanna fetch the datasheet of Vishays E102Z film resistors from here: http://www.vishaypg.com/foil-resistors/list/produc...

They can be had down to 0.005% tolerance (with a tempco of 0.2ppm/°C and the joy doesn't stop here.

Their Burden-Z resistors has got impressive data as well, but only goes to 500 Ohms max - the datasheet is a leisure to read anyway - for anyone with even a remote interest in electronics.

It can be found at: http://www.vishaypg.com/doc?63258

Please know, that I'm not trying to bash your posts in any way, I find them very informative and whether your meter is off or not, it will still be good data, if viewed as relative rather than exact values - even a non-adjusted meter will be precise enough for relative data, as the spread of values are so small. So please just consider it as FYI.

Thanks for posting it - while I don't care for the spread in 5% resistors from a design stand (if I need better, I specify better), it's definitely nice to get a brush up on the statistics behind :)

[JRV31]

That sounds interesting. It would be fun to do a calibration check on my meter. I have to decide if I want to spend $14 apiece on those resistors.

[Omnivent]

Ouch! I didn't see a price for them - sure sounds a bit steep. Personally, I wouldn't bother and most of the people that find it ever so important to have a meter adjusted to µV precision are just kidding themselves ;)

[Battlespeed]

Saying nothing about other deficiencies in your methodology, we'll just start with the fact that your sample is invalid.

[Marbie25]

This was clearly a study done out of personal interest - methodology does not need to be perfect. In fact, I guarantee you many published scientific studies are not perfect due to any number of "deficiencies". The point of any study is to make it as good as you can with the resources and time available.

All these types of comments (criticism without the constructive part) do is make people second guess their want to explore science.

Keep on, JV!

[JRV31]

Thank you.

[JRV31]

Why do you say my sample is invalid? I pulled the pieces at random from a bin of hundreds.

[cpetito]

I can't comment on your SPC methodology - I didn't do very well in my college statistic class, but to comment on the sample being invalid, unless the resistor manufacturing process has changed over the years, the resistors are manufactured, tested, sorted and then marked for tolerance. Therefore there is a very real possibility that your sample is not random.

This brought to mind a co-worker that wanted to get close to a particular resistor value. However all we had in the lab were +-10% resistors. He figured there was a possibility that there might be one close to the value he wanted and measured them all. It turned out that non of them where within +-5% - because they would have been marked and sold as such.

Thank you for the presentation. Perhaps I should see if age has improved my statistical comprehension. I wonder what the probability of that is!

[JRV31]

It was as random as I could make it given a bin with 300+ resistors in it. I don't know if they were all from the same manufacturing lot, or different ones.

[dmwatkins]

Good presentation! A complete error analysis would also require the tolerance and accuracy of the multimeter as well as changes in the testing environment conditions. Excellent though!

[Ciege666]

It's also worth mentioning that the accuracy of your multimeter should be taken into consideration as well as how long is been since it's last calibration check.