Choosing an approach and a model

Decide, before building anything, which kind of technology fits each part of a football performance analysis, and which language model to start with.

Every option has strengths and costs: fixed rules are predictable but rigid, language models read free text well but can be inconsistent or wrong, and cost and delay differ between models. Choosing by habit or by hype leaves those trade-offs unrecorded.

I compared four approaches (deterministic rules, machine learning, a language model and a hybrid) across ten criteria, such as flexibility, consistency, ease of validation, operating cost and risk of incorrect analyses. Then I compared three language models (GPT-4o, Claude Sonnet and Llama 3 70B) on quality, consistency, latency, cost and fit for the case. Both comparisons use qualitative levels and 1-to-5 scores that I assigned in a planning exercise.

A decision, not a measurement: a hybrid design, with rules for calculations and objective criteria, machine learning where classification is needed and a language model to interpret and write analyses, always behind output validation and human review for critical cases. For the pilot the model chosen was Claude Sonnet, for its balance of quality, consistency and moderate cost. No scores are published here, because they were judgments, not measurements.

The levels and scores are qualitative and have no stated measurement method, so they must not be read as a model ranking. The comparison was made at planning time, before the experiments on this page existed: those run on gpt-4o-mini, not on the model chosen for the pilot, and the scores were never checked against them. Cost and latency depend on the provider, the date and the infrastructure.

Writing the trade-offs down makes the choice open to discussion. The experiment on validating model output is one way of applying the hybrid idea: rules and a verifier around the model, because the model alone does not guarantee consistency.