Over the past few days, I had been refining the Hair Segmentation pipeline, and the False Positives around the jawline and cheeks had been reduced significantly. But as one problem disappeared, another became impossible to ignore.

From the front view, the eyeglass frames were still being included in the Hair Mask. From the side view, the eyebrow closest to the camera was also being recognized as hair.

At this point, it no longer felt like a simple parameter tuning problem. It was time to step back and understand the entire segmentation pipeline again.

“Did MediaPipe Actually Think the Glasses Were Hair?”

Me

“MediaPipe Hair Segmentation is trained to recognize hair. So why would it classify eyeglass frames as hair?”

Claude

“There are two possibilities.

The pixels may have already been classified as hair during the inference stage, or they may have been added later during post-processing.”

Until then, I had been treating every False Positive as the same problem. Claude’s answer made me realize that they could come from completely different stages of the pipeline.

If the model itself had already decided that the glasses were hair, no amount of post-processing would change the result.

On the other hand, if post-processing was introducing the error, then the model wasn’t the real problem.

The output looked identical, but the root cause—and therefore the solution—could be completely different.

The next idea that came to mind was texture.

Me

“Hair has texture, while eyeglass frames are relatively smooth and uniform. Could texture be used to distinguish between them?”

Claude agreed that it was a reasonable direction to explore.

After running several tests, however, an unexpected problem appeared.

The front view looked slightly better, but from the side, the eyebrow closest to the camera was still being detected as hair.

“Eyebrows Are Hair, Too.”

Me

“If we’re using texture, why is the eyebrow still being classified as hair?”

Claude

“Because eyebrows are real hair.

Humans distinguish them using facial structure and context, but the model doesn’t perceive the world that way.”

That was the moment my first hypothesis collapsed.

Texture might help separate glasses from hair, but eyebrows and sideburns are still hair by nature.

Adding another rule wasn’t going to eliminate every False Positive.

Then Claude asked a question I hadn’t expected.

Claude

“Before changing the logic any further, we should verify that this part of the pipeline is actually being executed.”

That made me stop and think.

Until then, I had been focused entirely on adding new filters.

But the more important question was somewhere else.

Me

“If the model already classified those pixels as hair during inference, does that mean the code I’m modifying never even gets a chance to run?”

Claude

“That’s absolutely possible.

The first classification isn’t based on ‘dark colors look like hair.’

It’s based on whether the model recognizes visual patterns that resemble hair.”

That single sentence completely changed the direction of my debugging.

I kept introducing new ideas, but none of them would matter if they never reached the actual source of the problem.

Then another possibility emerged.

Claude

The false positives could be coming from the color-based post-processing that we added ourselves.

If that’s the case, the root cause—and the solution—are completely different.”

At that point, I realized I had been trying to fix a problem before proving where it actually originated.

Instead of asking, “How should I improve this algorithm?”, the better question became:

“Which stage of the pipeline is actually responsible?”

That shift in perspective was probably the biggest takeaway of the day.

Revisiting the Rendering Pipeline

While working on segmentation, I also decided to revisit the rendering pipeline.

There was still an issue where some hair strands stretched diagonally instead of naturally falling downward. At first, I assumed the direction clamp was responsible.

But after reviewing previous debugging sessions with Claude, it became clear that the problem wasn’t caused by the clamp alone.

I restored a gravity-related setting that had been temporarily disabled during an earlier experiment, and the strands immediately began falling much more naturally.

The clamp itself wasn’t the problem.

The real issue was the balance between multiple components inside the rendering pipeline.

It was interesting that while investigating segmentation, I ended up rediscovering the real cause of a rendering issue as well.