Counting objects on conveyor belts

This article comes out of my on-ground experiences of deploying AI and ML in factories between 2022 and 2025 in Thailand. The task sounds trivial and is asked for constantly: count the objects moving past a point on a conveyor belt. It is the input to yield reports, to line-balancing, to paying piece-rate workers, so it has to be right - not “95% and improving.”

The useful thing to note is that most of the work happens before any model is chosen. Answer three questions about the line and the solution usually picks itself. Answer them wrong and you spend six weeks retraining a detector to cope with bad lighting, when a lamp and a shroud would have removed the problem on day one.

Three questions to ask first

How do the objects arrive on the belt? Cleanly separated, touching end to end, or piled on top of each other? This single answer moves you between three very different systems. Separated objects are a signal-processing problem. Touching objects need an instance-level detector - something like YOLO - that can draw a box around each one and separate the run into instances. Objects stacked on each other cannot be counted from one camera at all: a top-down view of two stacked items looks exactly like one item. That case needs a second camera at a different angle and “stitching the count”, or a mechanical change upstream - a singulator, a vibratory feeder, a faster pull-away belt - to stop the stacking in the first place.

What is the belt’s throughput? How many objects cross a fixed point per second? This sets the frame rate, and through it the camera and most of the cost.

How many objects are across the belt at once? One lane, or three, or a wide belt with items scattered anywhere across it? A single lane means you never have to answer “is this the same object I saw last frame,” which is the entire job of multi-object tracking (MOT). Dropping MOT drops the compute budget by an order of magnitude, and it drops a whole category of failure, ID switches, with it.

For the rest of this article: objects cleanly separated, fewer than 10 per second, single lane. Harder cases are for later articles.

That combination has an answer worth stating plainly, because it is the thing people resist hearing:

For separated objects in a single lane under controlled lighting, the right model is no model. A threshold on a strip of pixels will beat a neural network on accuracy, latency, cost and the amount of time you spend maintaining/updating it.

Consistent lighting is what enables it. It turns a hard ML problem into a thresholding problem. Most of the engineering below is spent buying that consistency.

Choosing the camera

Two independent constraints, and they are often confused with each other. One sets frame rate, the other sets exposure time.

Frame rate: how many frames per object

You need several frames while the object covers the line, so that a debounce can tell a real object from one flickering pixel. If $L$ is the object’s length along the direction of travel and $v$ is belt speed, the object covers the line for $L/v$ seconds, giving

$$N = \text{fps} \times \frac{L}{v}$$

frames on the object. Now substitute the throughput. At $R$ objects per second the spacing between objects is $p = v/R$, and the fraction of that spacing filled by the object - call it the duty cycle $d = L/p$ - gives

$$N = \frac{\text{fps}}{R} \times d$$

Belt speed cancels. Frames per object depends only on frame rate, throughput and how tightly packed the objects are. Speeding the belt up does not cost you frames per object; adding objects does.

This is what makes a speed-up belt such a good deal. Running the belt faster at the same throughput stretches the spacing $p = v/R$, which pulls touching or stacked objects apart - and it costs you nothing in frames per object. The only bill comes due on exposure time, in the next section.

That formula also corrects a tempting mistake. At 10 objects/second and 60 fps you get 6 frames per object period, and that is the number people quote. But if the objects fill 60% of the spacing, only about 3.6 of those frames actually see the object and 2.4 see the gap. Three frames is thin for a debounce, and a single dropped frame becomes a miscount.

A workable rule of thumb: frame rate ≥ 10 × objects per second. At 10 objects/second that is 100 fps or better, not 60.

Exposure: how much motion blur

Separately, the exposure has to be short enough that the object barely moves during it. If the camera looks at a field of view $W_{\text{fov}}$ wide, imaged onto $W_{\text{px}}$ pixels, then each pixel covers $s = W_{\text{fov}}/W_{\text{px}}$ of real width, and for blur under one pixel:

$$t_{\text{exp}} \le \frac{s}{v} = \frac{W_{\text{fov}}}{W_{\text{px}} \cdot v}$$

A 300 mm field of view across 1456 pixels is 0.21 mm per pixel. At a 500 mm/s belt that is $t_{\text{exp}} \le 0.41$ ms, about 1/2400 s. Short exposures need light, which is the next section.

For a threshold on a strip, a little blur is survivable - it softens the object’s edges, and the debounce absorbs that. What a global shutter really buys is that every row of the strip is exposed at the same instant, so an edge is where it appears to be, and that you can crop the sensor to a narrow band and read it out much faster than the full frame. That crop is how you get past 60 fps on cheap hardware.

Lighting is the actual project

Everything above assumes the belt looks the same in frame 900,000 as it did in frame 1. It will not, unless you make it.

• Block the ambient light. A shroud or tunnel over the 300 mm the camera sees. Sun through a skylight moving across the afternoon is the single most common cause of a counter that “worked in testing.”
• Use DC-driven LEDs. Mains-driven lighting flickers at 100/120 Hz. Against a 100+ fps camera that beats into a slow brightness pulse frame to frame, and your threshold crosses on the pulse rather than on an object.
• Lock the camera down. Manual exposure, manual gain, manual white balance. Auto-exposure will helpfully re-normalise the object right back into the belt’s brightness.
• Do not hard-code the belt colour. Belts get dusty, and they get wiped clean at shift change. Track the empty-belt colour continuously and threshold relative to it, which the code below does.

The solution

Detection

Sample a thin band of pixels on a line perpendicular to the direction of travel. Compare every pixel in the band to the current estimate of the empty-belt colour. If enough of them differ by more than a threshold, an object is on the line.

Use two thresholds, not one: how different a pixel has to be to count as “not belt,” and what fraction of the band has to be “not belt” before the line counts as occupied. The second one is what makes a stray fibre or a fly harmless.

Counting

A state machine. Occupancy goes low → high → low, and the falling edge adds one to the count - but only if the high run lasted at least MIN_FRAMES. That debounce is the whole reliability story: it throws away single-frame blips, which are the dominant failure mode.

Code

The first three constants are pure geometry - where on the sensor the band sits. Everything else is a threshold.

import cv2
import numpy as np

LINE_X = 700           # column of the counting line, in pixels
BAND = 8               # half-width of the sampled band
Y0, Y1 = 250, 850      # how far the band spans across the belt
PIXEL_THRESH = 40      # how far from belt colour a pixel must be
COVER = 0.30           # fraction of the band that must differ
MIN_FRAMES = 3         # debounce: ignore runs shorter than this
ADAPT = 0.01           # how fast the belt-colour estimate follows drift

cap = cv2.VideoCapture("belt.mp4")

belt = None            # running estimate of the empty-belt colour
count = 0
frames_on = 0

while True:
    ok, frame = cap.read()
    if not ok:
        break

    band = frame[Y0:Y1, LINE_X - BAND:LINE_X + BAND].astype(np.float32)

    if belt is None:
        belt = band.mean(axis=(0, 1))          # first frame must be empty belt

    # per-pixel distance from the belt colour, across all three channels
    diff = np.abs(band - belt).max(axis=2)
    occupied = (diff > PIXEL_THRESH).mean() > COVER

    if occupied:
        frames_on += 1
    else:
        if frames_on >= MIN_FRAMES:
            count += 1
        frames_on = 0
        # only learn the baseline while the belt is empty
        belt = (1 - ADAPT) * belt + ADAPT * band.mean(axis=(0, 1))

print(count)

Three things worth pointing at.

frames_on doubles as the state - nonzero means the line is occupied - so no separate was_occupied flag is needed. The baseline updates only in the empty branch, which is what stops an object that stalls on the line from being slowly absorbed into the background. And ADAPT at 0.01 gives a time constant of roughly 100 empty frames, fast enough to follow dust and a wipe-down, far too slow to follow an object.

On the Pi, swap cv2.VideoCapture for Picamera2 and configure a cropped, high-frame-rate mode; everything below the capture line is unchanged. Develop against a recorded video first - you want to be able to replay the same 10,000 frames while tuning PIXEL_THRESH and COVER.

Worth being clear about, because the two get conflated. There are still two imports at the top of that file. What was dropped is the model - there are no weights, no training set, no inference, no GPU, and nothing to retrain when the product changes colour. What NumPy is doing here is arithmetic on an array: a subtract, an absolute value, and two comparisons.

The two libraries earn their place differently, though.

OpenCV is optional in production. Its only job above is cv2.VideoCapture, and Picamera2 replaces that on the Pi without needing OpenCV at all. It is a development-time convenience for replaying recorded video, not part of the deployed stack.

NumPy is not optional. The band holds 600 × 16 × 3 = 28,800 values, and at 100 fps that is 2.88 million values per second. A pure-Python loop costs on the order of 100 ns per element once interpreter overhead is counted, so that one comparison alone would eat something like 3 ms of a 10 ms frame budget - before capture, before the mean. NumPy pushes the same loop into compiled C and does it in tens of microseconds. Roughly two orders of magnitude, and it is the difference between comfortably keeping up and not keeping up at all.

So: pure logic, yes - the algorithm is a threshold and a state machine you could explain on a napkin. Pure Python, no. The vectorised array math is what makes something this simple fast enough to be worth doing.

Hardware and cost

A Raspberry Pi 5 with the Raspberry Pi Global Shutter Camera (IMX296, 1456×1088 at 60 fps full-frame, faster cropped) is the right size of machine. A Jetson is overkill for what is, per frame, one subtraction and one comparison over a few thousand pixels.

Do budget for the whole thing, though. The camera module alone is around $50, but it ships without a lens - it is a C/CS mount - and the lighting and mounting are not optional extras, they are the parts that make it work.

Item	Ballpark
Raspberry Pi 5, 4 GB	$60
Raspberry Pi Global Shutter Camera (IMX296)	$50
C/CS-mount lens (6 mm or 16 mm, by working distance)	$25
Official 27 W USB-C power supply	$12
32 GB A2 microSD	$10
DC LED bar light + 12 V driver	$40
Bracket, shroud, enclosure	$30
Opto-isolated GPIO output board - only if the count goes to a PLC	$10
Total	$237

Prices are indicative - treat the shape of the list as the useful part, not the numbers. The point is that the honest figure is a couple of hundred dollars, not the $50 the camera’s price tag suggests, and it is still an order of magnitude under any smart-camera product that does the same job.

The last row is the only conditional one. If the count merely has to reach a dashboard, push it over the network and drop that line. You want a PLC connection when something on the line has to act on the count - stop the belt, trigger a diverter, close out a batch - because that is work you do not want depending on wifi.

If you do wire to a PLC, the isolation is not optional. Pi GPIO is 3.3 V and is not even 5 V tolerant, while industrial I/O is 24 V, so a direct connection destroys the pin and usually the chip behind it. A plain level shifter fixes the voltage but still shares a ground with the PLC, and on a conveyor that is the actual problem: the two supplies sit at different ground potentials, and the belt is driven by a variable frequency drive, which is one of the electrically noisiest things in a factory. An optocoupler passes the signal across an air gap as light, so there is no shared ground for that noise to arrive on. Ten dollars to put an air gap in front of a $60 computer and an hour of line downtime.

How much RAM?

At least 2 GB, and then RAM is not the constraint worth thinking about. The Pi 5 sells in 1, 2, 4, 8 and 16 GB variants, and the temptation is to buy headroom. Let’s do the maths instead.

The sampled band is 600 rows × 16 columns × 3 channels. As float32 that is 113 KB per frame. A full sensor frame is 1456 × 1088 × 3 = 4.5 MB, and the camera stack holds only a handful of those in flight. The memory that actually dominates is not the images at all - it is the Python runtime underneath them. A deployed stack of Python, NumPy and Picamera2 is a couple of hundred MB resident, on top of roughly 150 MB for a headless Raspberry Pi OS Lite; add OpenCV during development and it is more. Call it 500 MB in steady state while you are still tuning, and less once deployed.

Measure it on your own build rather than trusting those numbers - they move a lot between OpenCV builds and OS versions. The ratio is the part that holds: the algorithm’s working set is barely 100 KB and the interpreter around it is three orders of magnitude larger. Nothing you do to the counting logic will show up in the RAM figure.

1 GB technically runs it, but leaves nothing for a package upgrade or a second process. 2 GB is the sensible floor and where I would stop. The one honest reason to go to 4 GB is a ring buffer for diagnosis: keeping the last 30 seconds of cropped band in memory so that a suspected miscount can be dumped to disk and inspected. At 100 fps, storing the raw 8-bit band rather than the float32 copy, that buffer is only about 90 MB, so even this fits in 2 GB - it is worth the upgrade only if you want the same ring buffer of full frames, which at 4.5 MB each is a different order of problem.

What actually binds is CPU and sensor readout keeping up at 100+ fps, not memory. Spend the money on the lens and the lighting, not further on RAM.

If you need it cheaper

If $237 is more than the line can justify, the compute is the wrong place to look - the Pi is only $60 of it. The camera, lens and lighting together come to $115, close to half the build, so that group is where any real saving has to come from. Keep the architecture and downgrade the parts instead:

Item	Full build	Budget build
Compute	Pi 5 4 GB, $60	Pi 4 2 GB, $45
Camera	Global Shutter IMX296, $50	Camera Module 3, $25
Lens	C/CS mount, $25	included with camera, $0
Lighting	machine-vision bar, $40	12 V LED strip and diffuser, $12
Power	27 W USB-C, $12	same, $12
Storage	32 GB microSD, $10	same, $10
Mounting	bracket, shroud, $30	same, $30
PLC output	opto-isolated board, $10	MQTT over the network, $0
Total	$237	$134

Cost came down to nearly half, and every part still sits at the belt. The Global Shutter camera needs a lens bought separately, while the Camera Module 3 ships with one integrated, so changing camera deletes two rows at once.

Note also which rows did not move. Power, storage and mounting are irreducible.

Three things to check before taking the budget column. All three cost margin rather than money, which is the trade you are making:

• The Camera Module 3 is rolling shutter. Survivable for a thin band, as above, but it loses the sensor-crop trick that gets you past 60 fps. Take it only at throughputs well under 10 per second.
• The cheap strip may not be bright enough. The exposure worked out above was 0.41 ms, and an exposure that short needs a great deal of light. A $12 strip is dimmer and less even across the band than a machine-vision bar, and cheap drivers often dim by flicking the LEDs on and off quickly - the exact flicker the DC-LED rule warns about. Check that you can actually hold 0.41 ms at a sensible gain before saving $28.
• The Pi Zero 2 W cannot move frames fast enough. The counting arithmetic would run on anything; the bottleneck is the path from sensor to memory. What you get when it cannot keep up is dropped frames, and those are worse than they sound: one frame lost mid-object can pull a run below MIN_FRAMES, and the debounce then discards a real object as though it were noise.

”Can we just use the PC we already have?”

This is the first question every plant manager asks, and it is a fair one - there is a computer in the office already, so why buy another. The answer is counterintuitive:

The Pi is not really a compute cost. It is the thing that lets you use a $50 camera.

The Global Shutter Camera is $50 because it is a bare CSI sensor module - no housing, no interface electronics, no networking. CSI is a short board-to-board bus: the ribbon is good for 30–50 cm, and a couple of metres at the very most before signal integrity goes. A CSI camera cannot reach the control room. It structurally requires a small computer bolted next to it.

So the question is really about which interface replaces CSI, and each option has a catch:

• A USB webcam fails on requirements, not budget. Rolling shutter, auto-exposure you often cannot fully disable, MJPEG compression, 30 fps. We need a locked exposure and a frame rate ten times the throughput.
• An IP camera over Ethernet is cheap and PoE gives 100 m on a single cable, but it hands you H.264. The encoder smooths and blocks exactly the edges being thresholded, and the artifacts move around frame to frame. It is actively hostile to a pixel-difference method. Variable latency and silent frame drops wreck the debounce timing too.
• A USB3 Vision industrial camera is genuinely good - global shutter, locked exposure, screw-lock connectors that survive vibration. But passive USB 3 tops out around 3 m. Active cables reach ~15 m, optical extenders 50 m and beyond for $100–200, which is more than the Pi ever cost.
• GigE Vision with PoE is the correct industrial answer: 100 m, power and data on one cable. The camera is several hundred dollars.

The rule that falls out: if the PC is within about 3 m of the belt and you will only ever instrument one line, a USB3 Vision camera into the existing machine is legitimate and saves you the Pi. Outside that narrow case, the Pi is the cheap option, because everything that spans distance costs more than it.

I would push back on the shared-PC part regardless of distance. That machine gets Windows updates mid-shift, gets software installed on it, gets rebooted by whoever needs it - and this counter feeds yield reports and piece-rate pay. Frame timing on a shared desktop is also far less deterministic, which matters when the debounce is counting frames. A $60 box doing exactly one job is the more reliable engineering, not the compromise. It scales linearly, too: five belts is five independent Pis, rather than five long cable runs into one machine whose failure stops all five lines.

And if budget is what is really driving the question, deleting the Pi is not even the best answer to it. The budget build above saves $103 - more than the Pi costs in the first place - and it does that without giving up a self-contained box at the belt.

Changes to the line

Almost none, which is the reason to like this approach.

The one thing that matters is contrast between the object and the belt. If they are close in colour, change one of them - a belt in the complementary colour is a cheap consumable and a much better investment than a better model. More contrast is directly more margin on PIXEL_THRESH.

Beyond that: mount the camera where the belt is flat and not where it bounces over a roller, and put the counting line where objects are already settled, not where they are still being dropped or nudged into place.

What can go wrong on a real line?

Two objects touching get counted as one. The fix is to use the run length, since a double is roughly twice as long as a single:

n = max(1, round(frames_on / FRAMES_PER_ITEM))
count += n

Calibrate FRAMES_PER_ITEM from recorded video rather than from the spec sheet. This works well for doubles and triples and degrades past that; a long run of touching objects is genuinely the other problem - the one that needs a detector.

An object parked on the line during a belt stop. The state machine handles this correctly by construction: nothing is counted until the line clears.

An object nudged back and forth across the line gets counted every time it crosses. This is real on lines with manual intervention. The fix is two lines a short distance apart and counting only on the A-then-B ordering, which gives direction for a few extra lines of code.

Slow drift - dust on the lens, a lamp aging, the belt darkening. The adaptive baseline covers most of it. The rest is caught by monitoring, not by cleverness: log the mean diff value on empty belt, and alert when it walks away from where it started.

Deploying to more than one line

For deploying the code and upgrades, you can SSH into one Pi. You will not want to do it for five Pis. The moment you have several, the deployment story stops being an afterthought and becomes the thing that decides whether a threshold tweak takes ten minutes or an afternoon. This is worth setting up before you need it, because the point at which it hurts is also the point at which you are least willing to stop and fix it.

The principle to start from: stop pushing code to Pis one at a time. Two approaches are worth knowing, and they solve different halves of the problem.

Ansible, for the config

Ansible pushes over plain SSH - no agent on the Pi, nothing to install at the belt. One inventory file lists the machines, one playbook says what should be true on each, and ansible-playbook -i inventory deploy.yml walks the whole fleet.

What makes it fit this particular problem is not the code deployment. It is that LINE_X, Y0, Y1, PIXEL_THRESH and COVER are different on every line, because every line has a different camera position and a different product. That per-machine configuration is the part that actually takes time when you manage Pis by hand, and Ansible’s host_vars is built for exactly it: the playbook is shared, the constants are per-host, and the file that says line 4 uses LINE_X = 640 is in version control rather than in someone’s memory.

The limit is that it is push-based. A Pi that is powered off during a deploy simply does not get it, and you find out from the summary line at the end - so re-running until the fleet is clean is part of the routine.

Containers, for the dependencies

The other half of the problem is that Picamera2, NumPy and the OS libraries underneath them drift apart across machines, especially on Pis imaged months apart. That drift produces the worst class of bug on this list: the counter behaves differently on line 3 than on line 1, and nothing in your code explains why.

Building a container image fixes it by shipping the whole userspace as one artifact. Build once for linux/arm64 with docker buildx, push to a registry, and each Pi pulls the same digest. Watchtower can poll the registry and restart on a new image, which turns “deploy” into “push a tag” and gives you rollback for free - repoint to the previous tag.

Two Pi-specific catches, both worth hitting on your desk rather than at the belt. The container needs the camera passed through explicitly (--device /dev/video0, and the /dev/dma_heap and /run/udev mounts Picamera2 wants), and the image has to be built for ARM64 - an image built on an x86 laptop without buildx will not run.

Combine both

For the best solution, you will need a combination of the two: Ansible to place per-line configuration and manage the host, a container to pin everything above the OS. Config in host_vars, code and dependencies in the image.

If you are picking one to start: Ansible, if the Pis are all on one site and you can reach them, because per-line config is the pain that arrives first. Containers, if the counter has already behaved differently on two machines - that symptom is dependency drift, and Ansible will not fix it.

Two good practices to follow:

• Have each Pi report its version. One line in the existing MQTT heartbeat, carrying the git SHA or image tag. Without it, “which lines are on the new threshold?” is a question you answer by SSHing into all of them, which is precisely what this section was trying to avoid.
• Roll out to one line first. Let a new version run a full shift on one line and reconcile the count. Only then deploy to the other lines. A bad parameter that stops every belt at once is a much worse afternoon than one that stops a single belt.

How do you know the count is right?

Do not ship on eyeballed frames. Record an hour of production video, hand-count it once, and hold onto that clip forever - it is the regression test for every parameter change afterwards.

Then, in production, reconcile against something the counter cannot see: a downstream check-weigher, the packing count, the next station’s tally. A counter that is 99.8% right and drifting is much worse than one that is 99% right and stable, and only reconciliation tells you which one you have.

Next article: what changes when the objects arrive touching, and where YOLO actually starts to earn its cost.