CALCULATORS / TOLERANCES, FITS AND METROLOGY

Gauge and encoder resolution budget calculator

Uniform quantization standard uncertainty of a gauge.

tolerances-gauge-resolution.napkin / editable

SCENARIO COMPARISON

Compare the numbers.

This source describes an analytical model. Its quantities and checks are the view.

Scenario	standardUncertaintymm	ratio
	0.00288675	20
	0.0023094	25

All calculated quantities are listed with the inputs. Switch to Source to inspect the equations.

SCENARIOS / CALCULATED VALUESSAME SOURCE

Change an input to see the consequences.

resolution ratio target20.00 ✓ passes

BASELINE INPUTS

resolution0.01 mmtolerance0.2 mm

CALCULATED

standardUncertainty0.0028868 mm

ratio20

Edits change the baseline source.

THE SOURCE

Edits below update the same model.

# Gauge and encoder resolution budget
# Scope: Uniform quantization standard uncertainty of a gauge.
# Status: illustrative worked example; engineering review pending.
# Inputs are example values. Checks are editable example requirements, not standards.

resolution = 0.01 mm
tolerance = 0.2 mm

standardUncertainty := resolution/sqrt(12) [mm]
ratio := tolerance/resolution

check resolution ratio target: ratio >= 10

scenario alternative: resolution = 0.008 mm

Calculated from your source · not saved

Baseline · Baseline

resolution ratio target20.00 ✓ passes

alternative · alternative

resolution ratio target25.00 ✓ passes

Illustrative first-order example. Checks are editable requirements, not design standards. The broader topic may require additional equations, data, and validation.

Formula and units

This example uses the following equations. Input names and units match the editable source above.

standard uncertainty: standardUncertainty = resolution/sqrt(12) [mm]
ratio: ratio = tolerance/resolution

Original worked example

These are the baseline values in the downloadable source. Edits above let you test your own values.

Given

resolution: 0.01 mm
tolerance: 0.2 mm

Calculated

standard uncertainty: 0.00288675 mm
ratio: 20

Assumptions and limits

Uniform quantization standard uncertainty of a gauge.

The requirement checks use the limits written in this example. Passing a check does not establish that the model covers every load, material behavior, or operating condition in a real design.

Download this .napkin example ↓

Take it further with your agent

Napkin keeps the inputs, equations, checks and assumptions together in one study. Your agent sets it up; you change the numbers.

Try with Claude ↗

codex 'Use Napkin (https://napkin.engineering/mcp, or the local Napkin plugin) to build a study: Gauge and encoder resolution budget. Inputs: measurement range; counts; required tolerance. Outputs: resolution; quantization contribution. Model scope: Separate manufacturing tolerances, uncertainty and numerical error. Read napkin_capabilities and the current snapshot. Ask me for the values that matter, label assumptions with units, author .napkin source, validate it and apply it with napkin_source using the current revision, then open the study. Add checks for the limits I supply. Keep unsupported analyses explicit.'

Claude needs the Napkin connector; Codex needs the Napkin plugin. Set up your agent ↗

The broader study

The topic brief below describes the broader scope. The worked example above states the particular calculation it implements.

Inputs: measurement range, counts, required tolerance
Outputs: resolution, quantization contribution
Modes: calculate, compare, sweep
Drawing: Tolerance interval diagram

EXECUTABLE WORKED EXAMPLE · FIRST-ORDER MODEL · CHECK THE SCOPE & ASSUMPTIONS