The study finds that artificial intelligence models open may be more expensive in the long run.

Photo of author

By [email protected]


Since more companies adopt artificial intelligence, choosing the model to go to is a great decision. Although open models may look cheaper in the beginning, a new study warns that these savings can evaporate quickly, due to the additional computing power it requires.

In fact, open source artificial intelligence models are burned through the resources of computing much more than their closed competitors when performing the same tasks, according to Ticket Posted Thursday by Nous Research.

The researchers tested dozens of artificial intelligence models, including Google and Openai closed systems, as well as open source models from Deepseeek and Magistral. They measured the amount of computing efforts required for all identical tasks across three categories: simple knowledge questions, mathematics problems, and logical puzzles.

To do this, use the number of symbols that each model is used to solve and answer questions as is the case with consuming computer resources.

The authors of the study wrote: “Open-weight models use 1.5-4 x distinctive symbols more than closed symbols-and even 10 x for simple knowledge questions-which sometimes makes them more expensive for each query despite the low costs of everything.”

Why are the symbolic competence

In artificial intelligence, the distinctive symbol is part of the text or data – it can be a word or part of a word or even numbering marks – used by models to understand the language. The models are treated and created one text at the same time, and therefore the more symbols they use, the greater the computing power and time required by the task.

Since most of the closed source models do not reveal the process of raw thinking or Cot Series (COT)The researchers measured the computing efficiency by calculating the symbols they used instead. Since the models are bills through the total distinctive symbols used in the process of thinking and removing the final answer, the distinctive symbols for completion work as an agent for the effort needed to produce a response.

This is an important consideration for companies that use artificial intelligence for many reasons.

The researchers wrote: “First, while hosting open weight models may be cheaper, this cost feature can be easily compensated if it requires more distinctive symbols for the cause of a specific problem.” “Second, increasing the number of symbols will lead to longer generation times and increase cumin.”

The closed models were the clear winners

The study found that open models constantly use more distinctive symbols of closed models for the same tasks, sometimes three times the number of simple knowledge questions. The gap has narrowed less than twice for mathematics and logic problems.

“Openai, GROK-4) improves fewer symbols to reduce costs, while open models (Deepseeek, QWEN) use more symbols, perhaps for better thinking.”

Among the open models, Llama-3.3-Sunotron-SUPER-49B-V1 was the most efficient, while the judiciary models are the most effective.

Openai models were also the most prominent. Both O4-MINI models and new Open Open weights models have shown a wonderful symbolic efficiency, especially in mathematics problems.

The researchers have noted that the Openai GPT -Ss models, with a series of the most important of them, could serve as a standard to improve the efficiency of the distinctive symbol in other open models.



https://gizmodo.com/app/uploads/2024/10/CharacterAI.jpg

Source link

Leave a Comment