Introduction: Do Resistors Really Hold 5% Tolerance?

About: Jack passed away May 20, 2018 after a long battle with cancer. His Instructables site will be kept active and questions will be answered by our son-in-law, Terry Pilling. Most of Jack's instructables are tuto…

I just bought a bunch of 560 Ohm gold band resistors from a local surplus store. They were a nickel each. Is it possible to make millions of parts, sell them so cheap, and maintain the +/- 5% tolerance specified by the gold band? The tool used to answer this question is Statistical Process Control (SPC).

You will need:

A digital multimeter

A bunch of resistors the same value. (I measured thirty)

The spreadsheet template located in step five.

Step 1: Statististical Process Control

To understand SPC it is necessary to understand the following terms:

Nominal is the exact measurement you are trying to achieve. But in reality nothing is ever perfect. There is variation in all processes, the Tolerance sets the limits of how much variation is acceptable. The Lower Specification Limit (LSL) and Upper Specification Limit (USL) are the limits of the tolerance.

The Mean is the arithmetic average of a set of values, or distribution. The Median is the point where half of the values are less and half are more. The Mode is the most common value. In an ideal situation the Nominal, Mean, Median, and the Mode will all be the same.You can get a rough estimate of how consistent your process is running by comparing them.

Process Capability (CP) is the measurement to determine if the process is capable of holding the tolerance allowed. To find the CP first you find the Standard Deviation. The Mean + ( Standard Deviation * 3) gives you the Upper Control Limit (UCL). Next find the Lower Control Limit (LCL), Mean - ( Standard Deviation * 3). The capability is the ratio of the specification limits over the control limits, (CP = (USL - LSL) / (UCL - LCL). If the CP equals one the control limits fit exactly within the specification limits.

You want it to be larger than one to give you some room for error. The reason for using +/- three standard deviations is because in a normal distribution 68.2% of the values will fall within 1 standard deviation. 95.5% will fall within 2 standard deviations, and 99.7% will fall within three. These figures are mathematical constants known as the Empirical Rule. As the amount of variation increases the standard deviation will also increase.

CPK is the measurement of how well centered the Mean is to the Nominal, if they are identical the CPK will equal the CP. More variation between the two in either direction will result in a lower CPK.

The standard most widely used in industry is a CPK of 1.3.

The math mentioned here gets complicated, but it is easy to estimate. With a normal distribution over the center half of the tolerance with the mean centered on the nominal your CPK will be approximately 1.3. An even distribution over the center half of the tolerance with the mean centered on the nominal will give you a CPK of approximately 1.1. If the CP and CPK both equal exactly one 99.7% of the parts will be within the tolerance. The other .3% will be bad.

Step 2: Capable and in Control

In this illustration both of the control limits (LCL and UCL, plus and minus three times the standard deviation) are within the specification limits (LSL and USL) meaning the process is capable.

The vertical line represents the nominal specification and the top of the curve represents the process mean.

The mean is well centered on the nominal so the process is said to be in control.

CP and CPK are equal.

Step 3: In Control But Not Capable

Here the control limits fall outside the specification limits, but the mean is well centered on the nominal.

The process is in control but not capable.

There is to much variation in the process. it must be improved so the control limits fall within the specification limits.

Again CP and CPK are equal.

Step 4: Capable But Out of Control

In this illustration the control limits would fall within the specification limits if the mean was centered on the nominal.

The process is capable but out of control.

Make an adjustment to bring the mean in line with the nominal.

Here the CPK is lower than the CP.

Step 5: Enter the Data and Get the Results

Download the two spreadsheets at the bottom of the step. 560Ohm.ods is my data, the same as shown here. SPC.ods is a blank SPC template you can use for this or any other SPC study.

Statistical Process Control
Capability Study Worksheet
Company AxMan
Order # 560 Ohm +/-5% resistor
USL - 588
LSL - 532
Sample
1 - 557
2 - 557
3 - 560
4 - 555
5 - 555
6 - 559
7 - 560
8 - 558
9 - 558
10 - 560
11 - 560
12 - 554
13 - 560
14 - 558
15 - 556
16 - 560
17 - 558
18 - 556
19 - 560
20 - 564
21 - 555
22 - 561
23 - 558
24 - 555
25 - 559
26 - 558
27 - 558
28 - 556
29 - 557
30 - 557
Nominal - 560.0000
Mean - 557.9667
Median - 558.0000
Mode - 558.0000
CP - 4.2036
CPK - 3.8983
LSL - 532.0000
LCL - 551.3057
UCL - 564.6277
USL - 588.0000
High Sample - 564.0000
Low Sample - 554.0000

The CP and the CPK are both a long way above 1.3, these resistors are within the specified tolerance with plenty of room to spare.

The resistors were purchased at Axman, a surplus store in St.Paul MN.
I have no financial interest in Axman, I am just a customer.

Explore Science Contest

Participated in the
Explore Science Contest