Career · Retrospective

From dashcam ADAS to LLM evaluation: one question for nine years

I have worked on three very different kinds of model: vision in dashcams, vision and time series on factory and railway hardware, and LLM agents for manufacturing analytics. Looking back, the hard part was the same in all three. Getting a model to produce an output was rarely the problem. Knowing when that output deserved to be acted on was.

Pittasoft, 2017 to 2022: an alert is a promise

At Pittasoft I led ADAS development for dashcam products: forward vehicle detection, traffic-light recognition and a Forward Vehicle Start Alarm, the alert that tells you the car ahead has moved off. I also built a driver monitoring system, training YOLO models to detect distraction and drowsiness.

A driver-facing alert is a promise. If it fires when nothing happened, the driver learns to ignore it, and then it is worse than useless on the day it is right. Detection accuracy on a test set is only part of the picture; what matters is how the alert behaves over a drive, and whether a person will still believe it the next time.

That argues for reading failure cases one by one before trusting any aggregate. Years later, in my own CabinSense project, a phone head that had probably never seen a phone label still scored about 0.99 at the 90th percentile on a clip with no phone in it. A number like that is only worth something once you go and find out why.

OpenSysNet, 2022 to 2025: trust on constrained hardware

At OpenSysNet I took on leading the team as well as building, as AI engineering manager, running delivery under SAFe. The projects were edge vision and industrial time series:

On the edge, trust has a second meaning: will the model still behave the same once it is squeezed onto the hardware it ships on? Every optimisation, whether pruning, quantisation or a smaller backbone, is a change to the model, and needs measuring like one. My INT8 post is a small version of that lesson: on my development CPU the "faster" format turned out to be a size win and a speed loss, and I have not yet measured it on target hardware.

Leading a team changes what verification means. You stop checking every line yourself, and you have to decide what evidence you will accept instead: the test, the benchmark, the screenshot, the reviewer who did not write the code.

eMoldino, 2025 to now: when the model talks back

At eMoldino I build LLM systems for manufacturing: a production-insight agent on AWS Bedrock and LangGraph over Snowflake data, and an MCP server exposing four analytics tools: ROI, run rate, capacity and cycle-time efficiency. I also built an LLM evaluation suite: more than 210 tests across four modules, using LLM-as-judge and DeepEval for faithfulness, hallucination and answer relevancy.

LLMs make the trust question harder in a specific way. A dashcam detector that is wrong is usually wrong in a way you can see. An LLM that is wrong is fluent, formatted and confident. It does not look wrong. Two of my recent posts measure that gap:

The same instinct applies outside LLMs. Where do recommender wins actually come from? asks which stage of a recommender earns its latency, and why its top feature appears to be partly an artifact of how I built the training data.

What stayed the same

Across the three jobs, a few rules survived every change of model type:

What changed

The models got bigger and the outputs got richer, from a bounding box to a paragraph. The cost of checking went up with them: you can count a missed detection, but judging whether a paragraph is faithful needs another model, and that model needs checking too. The engineering answer has stayed the same, though: measure the system on the decisions it actually makes, against the cheapest credible alternative, and be clear about what you did not test.

That is what this blog is for. Each post takes one version of "when should a model be trusted?" and answers it with a measured result, including the parts that did not work.

Full experience and projects: trohit920.github.io. Previous post: Running AI delivery under SAFe

← All posts