Does the foundation matter?
Customers have reasons to prefer one open foundation over another: its license, where it was built, its size, or simply what their security team has already reviewed. So before we built Philotic Inspect on one, we asked how much the choice costs.
We fine-tuned three open foundations with the same recipe on the same vehicle-damage data and scored them on the same 3,430 held-out areas, including 834 with no damage. Untuned, they are far apart. Tuned, they land within about two points of each other.
Results
Damage type, before and after tuning
Is there damage in this area, and which of seven types? Eight answers, including no damage.
- Qwen3-VL-8B, tuned80.8
- Qwen3-VL-8B, untuned53.8
- Ministral 3 14B, tuned78.7
- Ministral 3 14B, untuned25.7
- Gemma 4 12B, tuned78.5
| Foundation, tuned | Damage type | Part (1,133) | Clean areas flagged |
|---|---|---|---|
| Qwen3-VL-8B | 80.8% | 82.1% | 4.2% |
| Ministral 3 14B | 78.7% | 78.9% | 4.9% |
| Gemma 4 12B | 78.5% | 80.5% | 7.4% |
What it means
The domain layer does most of the work. Ministral 3 14B starts at 25.7% on damage type, partly because it calls 77.1% of clean areas damaged. After tuning it reaches 78.7% and flags 4.9% of clean areas. Qwen3-VL-8B goes from 53.8% to 80.8%.
That is the case for the three-layer stack. The foundation is a choice you can make on your own terms, because the curated data, labeling and evaluation in the Philotic layer carry the capability across foundations. We build Inspect on Qwen3-VL-8B by default because it leads by a small, significant margin, and we can build it on another foundation when a customer asks.
Methods
- RecipeA small adapter trained on each foundation from the same examples, with the same evaluation afterwards.
- Test3,430 held-out areas from public vehicle-damage photos, including 834 areas with no damage. Part accuracy uses the 1,133 areas where a person outlined the parts.
- StatisticsIntervals are 95%. Differences between foundations use paired tests on the same areas.
- CaveatThese are results for one task. We test candidate foundations again on each customer’s task before choosing.
The headline benchmark against Claude Sonnet 4.6 and Qwen3-VL-235B is in Introducing Philotic Inspect.