Skip to main content

Machine Vision in Material Handling: What It Does and How It Works

Machine vision gives a material handling system a way to see the part it is about to move: where the part is, which way it faces, whether it is the right part, and whether it is fit to pass to the next station. UTEC Industrial designs, engineers, machines, fabricates, and installs custom material handling systems for aerospace and heavy industry from its Spokane Valley, WA facility, integrating Allen-Bradley PLC and motion control with in-house CNC machining, heat treating, and stress relief. This article explains what a vision system does on a plant floor, how its imaging chain of lighting, lens, sensor, interface, and software produces a usable result, how a camera is calibrated to guide a robot, where deep learning helps, and why a production camera is not a safety device. Vision sits near the end of the build chain, design → engineering → parts machining → fabrication → assembly → weld fatigue → stress relief → drives → controls → tuning → monitoring, but it depends on every mechanical link before it.

What does machine vision actually do in a material handling system?​

A vision system turns an image into a decision or a coordinate that the handling equipment can act on. The Hornberg handbook's manufacturing chapter, in its section on 3D systems, lists applications under six headings. The handling examples after each heading are this article's, not the handbook's:

  • Identification. Reading a code, a part number, or a type feature so the line routes the part correctly, for example confirming which of several casting patterns has arrived on a foundry conveyor.
  • Completeness check. Confirming that every fastener, clip, or insert is present before an aerospace subassembly moves to the next station.
  • Object and pose recognition. Finding where a part lies and how it is oriented, so a robot or transfer mechanism can pick it without a hard fixture.
  • Shape and dimension measurement. Gauging a profile, a hole pattern, or a board's width and thickness in line.
  • Surface inspection. Detecting cracks, stains, knots, or coating defects.
  • Robotics. Supplying the robot with a corrected pick or place position.

The handbook's types-of-production section has two headings, discrete unit production versus continuous flow, and job-shop production versus mass production. As engineering reasoning, the first distinction matters for handling: a sawmill board line or a strip line is continuous flow, while an airframe fitting moving between stations is a discrete unit imaged when it stops or passes a trigger point. The handbook's temporal-interface section gives discrete motion production, continuous motion production, and line-scan processing their own subsections; as engineering reasoning, the production type shapes when and how the image is taken (Hornberg 2017, Ch. 10, §10.2 Application Categories, §10.8 Temporal Interfaces and §10.10.3 Applications).

Where does a vision system sit in the material handling build chain?​

Vision is part of the intelligence layer, the controls, tuning, and monitoring at the end of the chain, but, as engineering reasoning, its accuracy is set upstream. The handbook's mechanical-interfaces section has subsections on dimensions and fixation, working distances, position tolerances, forced constraints, additional sensor requirements, and additional motion requirements, followed by environmental conditions, reproducibility, and gauge capability. The mapping below onto earlier links in the build chain is engineering reasoning, not the handbook's structure:

  • Design and engineering fix the camera's working distance and field of view and decide whether the part is presented in a known position or found anywhere in a bin.
  • Parts machining sets the repeatability of the fixtures, nests, and datum stops that present a part to the camera. A nest that varies by more than the camera's measurement tolerance makes every downstream measurement uncertain.
  • Fabrication, weld fatigue, and stress relief determine whether the frame carrying the camera and the part stays dimensionally stable. A mounting frame that moves as residual weld stress relaxes, or that vibrates with a nearby drive, shifts the camera relative to the calibration it was commissioned with.
  • Drives, controls, and tuning decide when the image is taken, how the result reaches the PLC, and how the robot or transfer car uses it.

UTEC Industrial machines fixtures and mounting interfaces to tolerances of ±0.001 in, which is the kind of mechanical repeatability a vision measurement relies on. The handbook covers reproducibility and gauge capability within that same mechanical-interfaces section (Hornberg 2017, Ch. 10, §10.5 Mechanical Interfaces, §10.5.8 Reproducibility and §10.5.9 Gauge Capability).

What are the parts of a machine vision imaging chain?​

This article models a vision system as a chain of five parts, and, as engineering reasoning, the result is only as good as the weakest link:

  1. Illumination, which creates the contrast the software needs.
  2. Optics, the lens that forms the image, sets the field of view, and limits depth of field.
  3. The camera and image sensor, which convert light to pixel values with a characteristic noise, sensitivity, and dynamic range.
  4. The camera-computer interface, such as GigE Vision or Camera Link, which moves image data at the pixel rate the application needs.
  5. Image processing software, which segments, measures, matches, or classifies the image.

The Hornberg handbook follows broadly the same order, with chapters on lighting, optical systems, camera calibration, camera systems, smart cameras, camera-computer interfaces, and machine vision algorithms. Its algorithms chapter covers thresholding and segmentation, edge extraction, fitting of lines, circles, and ellipses, template matching, stereo reconstruction, and optical character recognition. For sensors, the handbook covers CCD and CMOS technology, CMOS shutter concepts, sensor sizes, a section on qualifying cameras and noise measurement, and a section on camera noise that includes fixed pattern noise. As engineering reasoning, one failure mode is to buy the camera first and treat lighting and optics as accessories; the chain is designed from the part and its contrast outward (Hornberg 2017, Ch. 3 to Ch. 9).

How are camera and image sensor specifications compared?​

Camera data sheets are only comparable when they are measured the same way. EMVA 1288 is the European Machine Vision Association's standard for the measurement and presentation of specifications for machine vision sensors and cameras; its stated aim is a unified method to measure, compute, and present specification parameters for cameras and image sensors used in machine vision. Release 4.0 became effective in June 2021. EMVA states that until Release 3.1, from December 2016, application of the standard with a simple linear model was limited to cameras with a linear response and without any pre-processing, and that Release 4.0 takes into account the rapid development of camera technology.

For a specifying engineer this has a practical consequence, and the rest of this paragraph is engineering reasoning. Two cameras quoted with the same resolution can differ widely in how much signal they deliver per unit of light and how much noise they add, and that difference decides whether a dark casting, a wet log, or a machined aluminum surface under strong reflection can be imaged reliably at the line speed. Asking for EMVA 1288 data, and noting which release the data was measured to, is the way to compare sensors on measured performance rather than on marketing figures (EMVA 1288 Release 4.0-2021).

How is camera resolution matched to the part and the field of view?​

As engineering practice, resolution is worked backwards from the smallest feature that has to be seen. The handbook's system-design chapter has subsections on camera type, field of view, camera sensor resolution, spatial resolution, measurement accuracy, calculation of resolution, and resolution for a line scan camera, in that order.

An illustrative calculation, with assumed numbers rather than values taken from any source, shows the idea. Assume a lumber line needs to see a board 48 in wide, and the camera sensor is 2,448 pixels across:

  • Spatial resolution = field of view ÷ pixels = 48 in ÷ 2,448 px = 0.0196 in per pixel.
  • A defect 0.10 in across then covers 0.10 ÷ 0.0196 = about 5 pixels.

Assumptions: the full 48 in width is imaged by one camera with no margin, the lens adds no distortion, and the part stays at the focused working distance. In practice a margin is added for part position tolerance, and if 5 pixels is too few for the software to classify the defect, the options are a higher-resolution sensor, a second camera, or a narrower field of view per camera. For a continuous-flow line, the handbook gives line-scan resolution its own subsection; as engineering reasoning, resolution along the direction of travel also depends on line rate and conveyor speed. In the handbook's sequence, resolution is followed by the choice of camera, frame grabber, and hardware platform, and then by lens design, with subsections including focal length, lens diameter and sensor size, and sensor resolution and lens quality (Hornberg 2017, Ch. 2, Designing a Machine Vision System, pp. 36–43).

Why is lighting the first design decision in a vision system?​

As engineering reasoning, the software can only measure contrast that the lighting creates. The handbook's system-design chapter has a subsection headed "Concept: Maximize Contrast" under the choice of illumination, and its lighting chapter runs more than 100 pages (pp. 63–177), with sections on:

  • Lighting techniques, including diffuse and directed bright field, telecentric bright field, structured bright field, dark field, and transmitted (backlit) arrangements.
  • Light color and part color, including monochromatic, white, infrared, and ultraviolet light, and polarized light.
  • Light filters, including daylight suppression filters, neutral density filters, and polarization filters.
  • Lighting control, including brightness control and static, pulsed, and flash operation.
  • Suppression of ambient and extraneous light.
  • Lifetime, aging, and drift of light sources.

The failure modes in the rest of this paragraph are engineering reasoning. Sunlight through an open bay door or a skylight changes the image between morning and afternoon shifts. A light source that dims with age slowly pushes a threshold-based inspection toward false rejects. Radiant glow from a hot billet in a steel or aluminum mill, or steam in a pulp mill, adds light the system was not designed for. Flash illumination synchronized to the camera exposure, with filters matched to the light source, is one defense; the handbook's lighting-control section has subsections on static, pulse, and flash operation and on the suppression of ambient and extraneous light (Hornberg 2017, Ch. 2 p. 44 and Ch. 3 pp. 63–177).

When does material handling need 3D vision instead of 2D?​

As engineering reasoning, a 2D camera measures position and rotation in a plane; it cannot tell how high a part sits or how it is tilted. That is enough when parts arrive flat on a conveyor at a known height. It is not enough when parts are stacked, piled in a bin, or presented at varying heights, as castings, forgings, and machined blanks delivered in returnable containers can be.

The handbook's 3D section has one subsection titled "2.5D and 3D" and one titled "Point Clouds and Registration". It gives passive and active methods of 3D data acquisition their own subsections, and its algorithms chapter covers stereo reconstruction, including stereo geometry and stereo matching.

FANUC packages 3D detection for random bin picking. Its iRVision 3D Model Detection Function automatically generates the settings for detecting a part in various positions from the part's 3D CAD data, and FANUC says it can be applied to a system in which a robot picks up one of many parts placed randomly in a returnable container by using a 3D vision sensor. FANUC notes that the settings for each possible part position previously had to be made and registered manually, one by one. The detection processing runs on a PANEL iH Pro connected to the robot controller by an Ethernet cable (Hornberg 2017, Ch. 10, §10.10 3D Systems; FANUC Corporation 2021, iRVision 3D Model Detection Function).

How is a camera calibrated so a robot can use what it sees?​

A robot cannot use a pixel coordinate; it needs a position in its own coordinate frame. Getting from one to the other takes two calibrations:

  • Camera (intrinsic) calibration models the lens and sensor so pixel positions can be converted to geometric rays. Zhang's planar-pattern method requires the camera to observe a flat pattern shown at a few, at least two, different orientations. Either the camera or the pattern can be moved freely, and the motion need not be known. The method models radial lens distortion and uses a closed-form solution followed by a nonlinear refinement based on the maximum likelihood criterion.
  • Hand-eye calibration finds the position and orientation of the camera relative to the robot. Tsai and Lenz described computing the camera pose relative to the last joint of the robot in an eye-on-hand configuration from a series of automatically planned robot movements. In their system, each move took a total of 90 ms to grab an image, extract image features, and perform the camera's extrinsic calibration, and the hand-eye computation took about 100 + 64N arithmetic operations, where N is the number of stations. They also analyzed the critical factors influencing accuracy.

The handbook's robot-guidance case study lists calibration and communication as its two key points. As engineering reasoning, one failure mode is a camera bracket that is bumped, loosened, or replaced after commissioning: the images still look correct, but every pick is offset until hand-eye calibration is repeated. UTEC Industrial integrates FANUC robotic cells, including vision, with a FANUC design and engineering partner (Zhang 2000, IEEE TPAMI 22-11; Tsai and Lenz 1989, IEEE Transactions on Robotics and Automation 5-3; Hornberg 2017, Ch. 10, §10.11.5 Robot Guidance).

How well does deep learning handle bin picking?​

Deep learning can help where a part has too little texture or too much variation for classic template matching. Jiang and co-authors built a bin-picking system for textureless boxes that uses only depth images from an Intel RealSense SR300 camera mounted on the suction hand in an eye-in-hand arrangement. A deep convolutional neural network trained on 15,000 annotated depth images, generated synthetically in a physics simulator, predicts grasp points without first segmenting each object, and predicts the vacuum-cup grasp pattern for the two-cup hand. The reported results were:

  • A 97.5% pick success rate at speeds exceeding 1,000 pieces per hour, with a 7-degree-of-freedom robot on randomly posed boxes in clutter.
  • A grasp planning time of 0.034 s.
  • 79.16% of predicted grasp points within 30 pixels of the surface center, compared with 58.88% for Dex-Net 4.0.

The failure modes are just as instructive. Picks failed when the target box was adjacent to another box and the robot targeted the wrong box, and when the surface area was close to the size of the vacuum cup. The authors state that the current implementation can deal with only planar-faced objects with rectangular contours, because the training dataset was generated using only boxes, whose long and short surface-side lengths are required for class labeling. The rest of this paragraph is engineering reasoning. The same bounding lesson applies to inspection more broadly: a network trained on a defined set of surface or dimensional defect classes only recognizes those classes, so a new defect class is a new training-and-validation job, not a configuration change. The research was set in warehouse logistics, not a heavy plant, but the lesson transfers: a trained model is bounded by the part shapes it was trained on, and a new part family is a new validation job (Jiang et al. 2020, Sensors 20-3, 706, §7, §8.4 and §8.5).

How does a vision system connect to the PLC and the handling controls?​

The vision result is only useful if it reaches the controller at the right moment and in a form the controller can trust. The handbook's interface sections cover digital I/O, field buses, serial interfaces, standard Ethernet TCP/IP, OPC UA, and Ethernet-based real-time field buses, plus timing for discrete-motion, continuous-motion, and line-scan production. On an Allen-Bradley system the first three design details below come from Rockwell Automation's controller documentation; the fourth is engineering practice:

  • Event-driven logic. Logix 5000 controller tasks can be configured as continuous, periodic, or event. On ControlLogix 5580 and CompactLogix 5380 controllers, event tasks can be triggered by a consumed tag, an EVENT instruction, a change in module input data, or a motion event; the Armor CompactLogix 5370 table lists consumed tag, EVENT instruction, and motion events only. An event task executes immediately when its event occurs, so, as engineering reasoning, a camera result can start a sequence without waiting for the next scan of a continuous task.
  • Handshaking. Produced tags continually transmit based on the requested packet interval, so it can be difficult to know when new data has arrived. Rockwell Automation's guideline is to use a programmatic handshake: a bit can be set or a counter incremented in the produced tag to show the consumer that new data is present, and a return handshake via a reverse produced/consumed tag can tell the producer that the consumer received and processed it. As engineering reasoning, one failure mode without that handshake is a PLC acting twice on one stale pick coordinate.
  • Data integrity. Rockwell Automation's guideline is to use a CPS instruction to buffer produced and consumed data; CPS instructions provide data integrity for data structures greater than 32 bits, and the controller inhibits all interrupts while it executes one. As engineering reasoning, that matters for a multi-value pose result.
  • Sensing around the camera. Part-present sensors, encoders on the conveyor or transfer car, and triggers from the drive tell the camera when to image and tell the PLC where the imaged part has moved to.

UTEC Industrial, a Rockwell Automation Recognized System Integrator, programs Allen-Bradley ControlLogix and CompactLogix controllers over EtherNet/IP for the handling systems it builds (Hornberg 2017, Ch. 10, §10.7 Information Interfaces and §10.8 Temporal Interfaces; Rockwell Automation 1756-RM094N-EN-P-2025, Ch. 2 pp. 17–18, Ch. 4 p. 30, Ch. 5 pp. 39 and 41, and Ch. 8 p. 79).

Can a machine vision camera be used as a safety device?​

Not unless it is built and tested as safety equipment. IEC 61496-1:2020 sets general requirements for the design, construction, and testing of non-contact electro-sensitive protective equipment (ESPE) designed specifically to detect persons or parts of persons as part of a safety-related system. It does not deal with requirements for ESPE functions not related to the protection of persons, such as using sensing unit data for navigation, and it notes that applications such as protecting machinery or products from mechanical damage can call for different requirements. As engineering reasoning, a production camera that finds parts is not designed specifically to detect persons, so it is not a substitute for a safety light curtain or laser scanner that meets the standard.

Where a light curtain or scanner guards a vision-guided robot cell, ISO 13855:2024 specifies requirements for positioning and dimensioning safeguards, including the detection zones of ESPE, with respect to the approach of the human body or its parts towards hazards. It does not consider approaches such as running, jumping or falling, is not applicable to hazards from emissions or to failure of mechanical parts or gravity falls, applies to safeguards for the protection of persons 14 years and older, and its separation distances do not apply to safeguards used solely for presence sensing. ISO 13849-1:2023 specifies a methodology and provides related requirements, recommendations and guidance for the design and integration of the safety-related parts of control systems that perform safety functions, such as the parts that act on the safeguard, in high demand and continuous modes of operation. ISO 12100:2010 specifies principles of risk assessment and risk reduction for machinery design, and IEC 60204-1:2016+AMD1:2021 applies to the application of electrical, electronic and programmable electronic equipment and systems to machines not portable by hand while working. As engineering reasoning, one failure mode is a vision "presence check" quietly promoted to a personnel interlock (IEC 61496-1:2020; ISO 13855:2024; ISO 13849-1:2023; ISO 12100:2010; IEC 60204-1:2016+AMD1:2021).

What must happen before someone cleans or adjusts a camera inside a cell?​

Lenses and lighting windows in a foundry, a sawmill, or a biomass plant can collect dust and need cleaning, and a cleaning trip can put a person inside the robot or conveyor envelope. OSHA's lockout/tagout standard covers the servicing and maintenance of machines and equipment in which the unexpected energization or start up of the machine, or release of stored energy, could cause injury to employees. Servicing during normal production operations is covered only if an employee must remove or bypass a guard or other safety device, or place any part of the body into the point of operation or an associated danger zone during a machine operating cycle; minor tool changes, adjustments and other minor servicing that are routine, repetitive, and integral to production are not covered if alternative measures give effective protection. As engineering reasoning, a cleaning trip into a robot or conveyor envelope meets the danger-zone condition. The standard defines push buttons, selector switches, and other control-circuit-type devices as not being energy-isolating devices, so stopping the robot from the HMI or opening the camera's software does not isolate the cell. Under 1910.147(d)(5)(i), after lockout devices are applied, all potentially hazardous stored or residual energy must be relieved, disconnected, restrained, and otherwise rendered safe. As applied here, in a vision-guided cell that stored energy includes a robot arm holding a part under gravity, and pneumatic pressure in a gripper or vacuum system.

The handbook's remote-maintenance section for vision systems carries a safety precaution headed "No Movements." As engineering practice, a good design places cameras and lights where they can be reached from outside the guarded zone, or provides windows that can be cleaned without entry, so routine cleaning does not require entry into the guarded zone (OSHA 29 CFR 1910.147-1989, (a)(1)(i), (a)(2), (b) and (d)(5)(i); Hornberg 2017, Ch. 10, §10.9.3.1 Safety Precaution: No Movements).

How is a vision system tuned, validated, and monitored after startup?​

As engineering reasoning, a vision system commissioned on a clean day with new lights is not proven until it has run through real production variation. The handbook's project-realization section covers development and installation, a test run and acceptance test, and training and documentation, and its calibration chapter includes verification of calibration results. The list below is engineering practice; the handbook is cited only for the topics it names (light-source aging and drift, reproducibility and gauge capability, traceability, and result data):

  • Acceptance on production parts. The test run uses parts across the real range of color, surface finish, temperature, and position, not a single golden sample.
  • Gauge capability. Where the vision system measures, its repeatability is checked the same way as any other gauge, which the handbook covers under reproducibility and gauge capability.
  • Drift monitoring. Light source aging and drift, lens contamination, and camera mount movement all show up as a slow change in image brightness, contrast, or match scores. Trending those values flags a problem before the reject rate climbs.
  • Traceability and result data. Storing results and images per part supports root-cause work when a defect escapes.
  • Recalibration triggers. Any bracket, lens, or robot tool change is a trigger to verify calibration before production resumes.

UTEC Industrial performs factory acceptance testing and on-site commissioning, so acceptance criteria for a vision-equipped handling system can be written into the purchase order and demonstrated before handover (Hornberg 2017, Ch. 2 p. 49, Ch. 3 p. 88, Ch. 5 p. 308 and Ch. 10 §10.7.2).

What should a specifying engineer define before buying a vision system?​

The handbook's section on specifying a machine vision system has subsections on task and benefit, parts, part presentation, performance requirements, information interfaces, installation space, and environment, and ends with a checklist. As engineering practice, those headings give a structure for a request for quotation, and the descriptions below are this article's. A complete vision specification for a handling system defines:

  • Task and benefit: what decision the system makes and what it saves, such as fewer mis-picks or fewer escaped defects.
  • Parts: the part types, their variation in color, finish, and geometry, and how each is presented to the camera.
  • Performance requirements: measurement accuracy and time performance, meaning the cycle time or line speed the result must keep up with.
  • Information interfaces: what data goes to the PLC or robot, in what format, and over which network.
  • Installation space and environment: mounting space, working distance, ambient light, temperature, dust, moisture, and vibration.

As engineering reasoning, two failure modes follow when this list is skipped. The first is a system that works on the bench but misses cycle time on the line, because time performance was never specified. The second is a system that meets accuracy with a perfect sample but fails on the real spread of parts, because part variation was never described. Writing both into the specification lets the supplier size the lighting, sensor, and processing before anything is built (Hornberg 2017, Ch. 2, Specifying a Machine Vision System, pp. 32–35).

Related Articles

References​

  • Hornberg, A. (Ed.). Handbook of Machine and Computer Vision: The Guide for Developers and Users, 2nd ed. Wiley-VCH, 2017. ISBN 9783527413393
  • EMVA 1288 Release 4.0-2021: Standard for Characterization of Image Sensors and Cameras (Release 4.0 Linear). European Machine Vision Association, 2021.
  • FANUC Corporation (2021). "ROBOT New Function: iRVision 3D Model Detection Function." FANUC News, 2021.
  • Zhang, Z. (2000). "A flexible new technique for camera calibration." IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), 1330-1334. DOI 10.1109/34.888718
  • Tsai, R. Y., & Lenz, R. K. (1989). "A new technique for fully autonomous and efficient 3D robotics hand/eye calibration." IEEE Transactions on Robotics and Automation, 5(3), 345-358. DOI 10.1109/70.34770
  • Jiang, P., Ishihara, Y., Sugiyama, N., Oaki, J., Tokura, S., Sugahara, A., & Ogawa, A. (2020). "Depth Image-Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking." Sensors, 20(3), 706. DOI 10.3390/s20030706
  • Rockwell Automation 1756-RM094N-EN-P-2025: Logix 5000 Controllers Design Considerations. Rockwell Automation, 2025.
  • IEC 61496-1:2020: Safety of machinery — Electro-sensitive protective equipment — Part 1: General requirements and tests. IEC, 2020 (Ed.4).
  • ISO 13855:2024: Safety of machinery — Positioning of safeguards with respect to the approach of the human body. ISO, 2024.
  • ISO 13849-1:2023: Safety of machinery — Safety-related parts of control systems — Part 1: General principles for design. International Organization for Standardization, 2023.
  • ISO 12100:2010: Safety of machinery — General principles for design — Risk assessment and risk reduction. ISO, 2010.
  • IEC 60204-1:2016+AMD1:2021 CSV (Ed. 6.1): Safety of Machinery -- Electrical Equipment of Machines -- Part 1: General Requirements. International Electrotechnical Commission, 2021.
  • OSHA 29 CFR 1910.147-1989: The Control of Hazardous Energy (Lockout/Tagout). Occupational Safety and Health Administration, 1989.

Ready to Discuss a Material Handling System?​

UTEC Industrial designs, engineers, machines, fabricates, and installs custom material handling systems for heavy industry, from the stress-relieved structure and drives to the Allen-Bradley PLC controls, tuning, and monitoring that run them, at its Spokane Valley, WA facility. Send UTEC the application, loads, and duty cycle to start a system review.

Request a Quote →

Questions? Call (509) 922-1832 or email sales@utec.co

Last updated on