I have been trying to run some very simple API calls using Claude to operate the new Barsukas HTTP API.
The flow is a 20-word prompt, reading 3-4 files from disk, and making a 4-line Python script to call the API.
This costs 22 cents. ⚙️ which feels like too much
Why so much? There is something like 8 KB of system prompt and 14 KB of system tools. All of which should be cached/built-in to the model, but is instead just charged to the user. And several rounds of "cached reads" as the tool pauses for inputs.
Fortunately, the $20/month subscription gives something like $300/month of credits. When the cost is inflated by a factor of 5, it helps that the currency is also inflated.
----
On the other hand, the sheer volume of struggles Claude Opus is having with this task (simple directions like "only read the api/ dir" get ignored about 1/4 of the time) does make me want to look for a different harness, which can pivot between model providers. ⚙️ it's not as bad as it was six weeks ago, but it is still very disappointing for the "recommended/expensive model" that it can't do this well.