One AI layer for Study Buddy, quiz import and demo chat

Three AI features now call one shared layer. This page reports the local run: the same inputs on the old code and the new code, on five models, with the numbers and every answer.

SP-893 · pull request 3 of 6 · 22 Nov 2026

Status as of 22 Nov 2026

Stage
In review. Three PRs are open and stacked. Nothing is merged, nothing is on beta, nothing is in a production tag.
PR
#759 AI SDK upgrade and the spec. Open. Claude approved.
PR
#760 The shared AI layer and its usage record. Open, on top of #759. Claude approved.
PR
#762 The three features move to the shared layer. Draft, on top of #760. This page is about this PR.
Review
22 Nov 2026: Claude reviewed #762 and found no code defect. It does not approve until Khai Hoan approves the old and new answers.
Review fixes
22 Nov 2026: a local review found four items. All four are fixed and pushed. The model runs on this page are from 21 Nov 2026, before these fixes.
Ticket
SP-893
Data
There are no screenshots and no video, because the change is in the backend. All inputs are made-up samples. The names and numbers are not real.

Why this exists

Each AI provider had its own code in the backend. A new model often failed in one feature and worked in another. Study Buddy could stream its answer only on Gemini. A change of model needed code changes in several places.

The shared layer has the same code for every provider. A new model is one row in a list. This PR moves the first three features to it.

What changed, in one picture

Step 1

Feature

Study Buddy, quiz import or demo chat asks for an answer.

Step 2

Shared layer

One code path builds the request for the tenant's provider and model.

Step 3

Model

Gemini, Claude or OpenAI answers. Study Buddy gets the answer in pieces.

Step 4

Usage record

Each request writes one row with tokens and cost.

How the local run was done

  1. Made-up inputs

    Three courses for demo chat. One quiz with 10 questions, as a Word file and as a PDF. One lesson transcript with 17 lines for Study Buddy.

  2. Real feature code, real models

    Each feature ran as the real feature code on a real model. It ran one time on the old code and one time on the new shared layer.

  3. Five models

    Gemini with the default model name, Gemini 3.8 Flash, Claude Sonnet 5.5, Claude Haiku 5.5 and GPT-6 Luna. That is 37 runs.

  4. Saved for each run

    The answer, the time, the usage row and its cost. For quiz import, the answers were compared with the 8 known correct answers.

Results

"Old" is the code on main. "New" is the shared layer.

ModelPathDemo chatQuiz import, Word and PDFStudy Buddy stream
Gemini, default model nameOldWorksWorks. 8 of 8 answers rightStreams. 13 text events
Gemini, default model nameNewWorksWorks. 8 of 8 answers rightStreams. 16 text events
Gemini 3.8 FlashOldWorksWorks. 8 of 8 answers rightStreams. 13 text events
Gemini 3.8 FlashNewWorksWorks. 8 of 8 answers rightStreams. 13 text events
Claude Sonnet 5.5OldFails (503)FailsNo streaming (501)
Claude Sonnet 5.5NewWorksWorks. 8 of 8 answers rightStreams. 13 text events
Claude Haiku 5.5OldFails (503)Works. 8 of 8 answers rightNo streaming (501)
Claude Haiku 5.5NewWorksWorks. 8 of 8 answers rightStreams. 11 text events
GPT-6 LunaOldFails (503)Works. 8 of 8 answers rightNo streaming (501)
GPT-6 LunaNewWorksWorks. 8 of 8 answers rightStreams. 15 text events

Read every answer. Old and new answers, side by side. Khai Hoan must approve these before #762 can merge.

All runs, with time, tokens and cost

Cost is in USD, from the usage row. Reasoning tokens are the tokens the model used to think.

ModelFeaturePathResultTimeInputOutputReasoningCost USD
Gemini, default model nameDemo chatOldWorks. 282 characters6.1 s700944020.002385
Gemini, default model nameDemo chatNewWorks. 332 characters9.8 s7001037310.003652
Gemini, default model nameQuiz import, Word fileOldWorks. 8 of 8 answers right, 11 items25.6 s2207162621730.015558
Gemini, default model nameQuiz import, Word fileNewWorks. 8 of 8 answers right, 11 items11.4 s2207172600.008128
Gemini, default model nameQuiz import, PDF fileOldWorks. 8 of 8 answers right, 10 items28.0 s2855165117360.013955
Gemini, default model nameQuiz import, PDF fileNewWorks. 8 of 8 answers right, 11 items13.3 s2855172600.008614
Gemini, default model nameStudy Buddy, streamedOldWorks. 13 text events, 9 citations9.1 s10883814750.004026
Gemini, default model nameStudy Buddy, streamedNewWorks. 16 text events, 11 citations7.7 s10884294680.004180
Gemini 3.8 FlashDemo chatOldWorks. 344 characters8.5 s7001037860.003859
Gemini 3.8 FlashDemo chatNewWorks. 360 characters6.5 s7001074220.002509
Gemini 3.8 FlashQuiz import, Word fileOldWorks. 8 of 8 answers right, 11 items11.1 s2207139400.006539
Gemini 3.8 FlashQuiz import, Word fileNewWorks. 8 of 8 answers right, 11 items9.5 s2207139400.006528
Gemini 3.8 FlashQuiz import, PDF fileOldWorks. 8 of 8 answers right, 11 items14.6 s2855110300.005390
Gemini 3.8 FlashQuiz import, PDF fileNewWorks. 8 of 8 answers right, 11 items11.8 s2855172600.007719
Gemini 3.8 FlashStudy Buddy, streamedOldWorks. 13 text events, 9 citations8.0 s10883935970.004529
Gemini 3.8 FlashStudy Buddy, streamedNewWorks. 13 text events, 9 citations8.2 s10884104790.004150
Claude Sonnet 5.5Demo chatOldFails (503)2.6 s0000
Claude Sonnet 5.5Demo chatNewWorks. 468 characters3.9 s11142851790.006868
Claude Sonnet 5.5Quiz import, Word fileOldFails0.7 s0000
Claude Sonnet 5.5Quiz import, Word fileNewWorks. 8 of 8 answers right, 11 items8.2 s3864154000.023128
Claude Sonnet 5.5Quiz import, PDF fileOldFails0.4 s0000
Claude Sonnet 5.5Quiz import, PDF fileNewWorks. 8 of 8 answers right, 10 items7.1 s7032138500.027914
Claude Sonnet 5.5Study Buddy, streamedOldNo streaming (501)-----
Claude Sonnet 5.5Study Buddy, streamedNewWorks. 13 text events, 11 citations4.0 s195459500.009858
Claude Haiku 5.5Demo chatOldFails (503)0.0 s0000
Claude Haiku 5.5Demo chatNewWorks. 458 characters4.9 s1114272443no price yet
Claude Haiku 5.5Quiz import, Word fileOldWorks. 8 of 8 answers right, 10 items4.5 s400814090no price yet
Claude Haiku 5.5Quiz import, Word fileNewWorks. 8 of 8 answers right, 10 items4.8 s386413850no price yet
Claude Haiku 5.5Quiz import, PDF fileOldWorks. 8 of 8 answers right, 10 items4.5 s717614090no price yet
Claude Haiku 5.5Quiz import, PDF fileNewWorks. 8 of 8 answers right, 10 items5.0 s703214560no price yet
Claude Haiku 5.5Study Buddy, streamedOldNo streaming (501)-----
Claude Haiku 5.5Study Buddy, streamedNewWorks. 11 text events, 11 citations4.9 s1954635532no price yet
GPT-6 LunaDemo chatOldFails (503)2.1 s0000
GPT-6 LunaDemo chatNewWorks. 206 characters4.7 s668691240.000163
GPT-6 LunaQuiz import, Word fileOldWorks. 8 of 8 answers right, 10 items8.0 s22697931290.000484
GPT-6 LunaQuiz import, Word fileNewWorks. 8 of 8 answers right, 10 items7.4 s2276791580.000709
GPT-6 LunaQuiz import, PDF fileOldWorks. 8 of 8 answers right, 10 items7.6 s24958011110.000481
GPT-6 LunaQuiz import, PDF fileNewWorks. 8 of 8 answers right, 10 items7.9 s231184700.000712
GPT-6 LunaStudy Buddy, streamedOldNo streaming (501)-----
GPT-6 LunaStudy Buddy, streamedNewWorks. 15 text events, 9 citations4.6 s1080296590.000189

Claude Haiku 5.5 shows "no price yet" because this run was before its price was added. A later run of Haiku 5.5 with the price is in the hand check below.

Why it can be trusted

Checked by hand. Claude Sonnet 5.5, demo chat: 1,114 input tokens at 2.00 USD for a million is 0.002228 USD. 464 output tokens (285 text and 179 reasoning) at 10.00 USD for a million is 0.004640 USD. The sum is 0.006868 USD, the same as the usage row.

Claude Haiku 5.5, a later run with the price: 1,114 input tokens at 0.10 USD for a million is 0.0001114 USD. 572 output tokens at 0.50 USD for a million is 0.000286 USD. The sum is 0.0003974 USD, the same as the usage row.

What the reviews found, and the fixes

FindingFix
The Study Buddy screen still asked for a streamed answer only on Gemini. Tenants on Claude or OpenAI got the answer in one piece.The backend now reports streaming for every provider. A test covers it.
A gateway (a third-party endpoint for models) got a thinking setting that the old code never sent. Some gateways refuse it.A gateway gets no thinking setting, as before.
In a list of several models, a gateway could get a PDF that it cannot read.A gateway never gets a file. Quiz import sends the text of the PDF.
No test covered quiz import when a model cannot read the PDF.A test now covers the change to the text of the PDF.
Earlier, on #760: gateways lost the change to PDF text.Fixed in #762 before this review.

Known limits

Decisions taken

DecisionChoice
Who approves a change in AI answers?Khai Hoan reads old and new answers side by side before the merge.
How do the two new Claude models start?Marked beta in the list until their answers are reviewed.
What does a gateway get?Text only, no file and no thinking setting, as with the old code.
When are the old provider adapters deleted?In pull request 4, after this comparison is approved.

Rollout checklist

  1. Khai Hoan reads the old and new answers and approves or sends comments.
  2. Merge #759, then #760, then #762. No migration and no new config.
  3. On beta: run Study Buddy, quiz import and demo chat on a real lesson and a real quiz. Compare the usage rows with the old rows. Switch a tenant to Claude Sonnet 5.5 and Haiku 5.5 in both pickers.
  4. Move SP-893 to Testing after all six PRs are on beta, and to Done once it is in a production tag.