Do AI inference models require new approaches to motivation?

The era of artificial intelligence is well underway.

After OpenAI once again started the AI ​​revolution through o1 Inference model Introduced back in September 2024 – which takes longer to answer questions but rewards high performance, especially on complex, multi-step problems in mathematics and science – the commercial AI field has been filled with imitators and competitors.

there Deep Sec R1, Google Gemini 2 Flash ThinkingAnd today only, LlamaV-o1all of which seek to offer built-in “inference” similar to OpenAI’s new o1 and upcoming o3 model families. These models are involved in Claim a “Chain of Ideas” (CoT). – or “self-prodding” – forcing them to think about their analysis halfway through, go back, check their work and ultimately come up with a better answer than just launching it from outside their head. Implications as quickly as possible, as do other large language models (LLMs).

However, the high cost of o1 and o1-mini ($15.00/million input tokens versus $1.25/million input tokens for GPT-4o) OpenAI API) caused some to reject the supposed performance gains. Is it really worth paying 12 times as much as a typical, advanced LLM degree?

As it turns out, there are a growing number of converts – but the key to unlocking the true value of inference models may lie in the user motivating them differently.

Xun Wang (Founder of Artificial Intelligence News Service Small) appeared on him Substack Over the weekend, a guest post from Ben Hylak, former interface designer for Apple Inc. For VisionOS (which powers the Vision Pro spatial computing headset). This post has gone viral because it convincingly explains how Hylak queries OpenAI’s o1 model to get incredibly valuable (to him) output.

In short, instead of a human user writing prompts for the o1 form, they should consider writing “summaries” or more detailed explanations that include a lot of context upfront about what the user wants the form to output, who the user is and the format in which they want the form to output information to them .

As Hillak writes Substack:

In most models, we are trained to tell the model how we want it to answer us. For example, “You are an experienced software engineer.” Think slowly and carefully“

This is the opposite of how I achieved success with o1. I don’t teach him how – just what. Then let o1 take charge and plan and decide his own steps. This is what independent thinking is for, and it can actually be much faster than if you were manually reviewing and chatting as the “human in the loop.”

Hylak also includes a nice annotated screenshot of a vectorized example for o1 that produced useful results for the flight listing:

This blog post was so helpful that OpenAI President and Co-Founder Greg Brockman reshared it on his X account with message: “o1 is a different kind of model. Great performance requires using it in a new way compared to standard chat models.

I’ve experienced this myself in my recurring quest to learn to speak Spanish fluently and Here was the resultFor those curious. It may not be as impressive as the responsiveness of the good, well-designed Hylak, but it certainly shows strong potential.

Separately, even when it comes to nonsensical LLMs like Claude 3.5 Sonnet, there may be room for casual users to improve their prompting for better, less restrictive results.

As Louis Argue, former Teton.ai engineer and current creator of the openFUS neuromodulation device, said, Written on X“One trick I’ve discovered is that LLMs trust their own claims more than mine,” he said, giving an example of how he convinced Claude to be “less cowardly” by “picking a fight” with him first. on its outputs.

All of this shows that agile engineering remains a valuable skill as the age of AI continues.

Daily insights into business use cases with VB Daily

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from organizational transformations to hands-on deployments, so you can share insights to maximize ROI.

Read our privacy policy

Thanks for subscribing. Check more VB Newsletters are here.

An error occurred.