vLLM keeps requested text-probability values in Cohere replies
By OnTheRice · Sources and limitations · Report a correction
vLLM accepted requests for estimated chances of generated text but omitted the values from some replies. The changed code keeps those values and text-unit identifiers. Streaming replies and a released version remain unverified.
Source confidenceNot assessed
What’s changing?
The response software now carries identifiers for generated text units and the model's estimated chances into the Cohere-style reply, rather than dropping them. The retained evidence covers non-streaming replies only.
Key points
- The Cohere wrapper accepted a log-probability request but omitted the values from non-streaming replies. The merged path now returns token IDs and probabilities.
What could change this view?
- Evidence is a focused unit test; streaming replies and a tagged release remain unverified.
Bottom line
The Cohere wrapper accepted a log-probability request but omitted the values from non-streaming replies. The merged path now returns token IDs and probabilities.
Its source facts and prices still need independent checks.
Terms in simple English
These explanations help you read this signal. The saved forecast, prices, deadline and outcome rules stay the same.
- Asset or investible
- Something that can be bought or sold, such as a share, bond, fund, currency or cryptocurrency.
- Probability
- An estimate of how likely an outcome is. For example, 60% means about six chances in ten, not certainty.
- API
- An application programming interface: a defined way for software services to request information or actions from each other.
How likely is this call? · Three frameworks
This signal has no investible bullish or bearish call. A market-direction percentage does not apply.
Evidence behind this view
Checks whether separate pieces of evidence support the same view.
Recorded connection
The response adapter carries the generated token metadata into the Cohere-shaped result instead of dropping it.
We have not established a separate count of independent evidence clusters for this card. Several reports may rely on the same event or data provider.
What past price moves show
Compares today’s price moves with recorded past patterns.
This card has no recorded investible call. A price-pattern comparison is not applicable unless a specific asset is added with evidence.
Evidence against this view
Shows evidence and risks that could challenge this view.
Recorded opposing evidence and risks
Evidence is a focused unit test; streaming replies and a tagged release remain unverified.
Why the direction changed
Explains recorded changes from expecting a rise to expecting a fall, or the reverse.
No documented bullish-to-bearish or bearish-to-bullish change is available for this card. We keep the original call and its time unchanged.
What people say vs the facts
Compares recorded public discussion with checked market facts.
No separate Reddit or public-discussion comparison with verified market data is recorded for this card. We cannot treat social attention as proof of a market move.
How past forecasts performed
Shows recorded forecast results and what they can teach us.
This card has no recorded investible forecast to score.
No tested mapping from evidence combinations to future success is recorded for this card. Confidence in sources and past outcomes do not establish a calibrated probability.
Sources, original wording and price checks
Author-submitted research. The underlying facts and market prices have not been independently verified.
Original research record
Observed change: The Cohere wrapper accepted a log-probability request but omitted the values from non-streaming replies. The merged path now returns token IDs and probabilities.
Mechanism: The response adapter carries the generated token metadata into the Cohere-shaped result instead of dropping it.
Uncertainty: Evidence is a focused unit test; streaming replies and a tagged release remain unverified.
Next check: Compare live streaming and non-streaming replies in the first containing release.
Published
First published through Sites 07 Oct 2026, 18:40:42 SGT
No explicit investible BULLISH or BEARISH call on this card.
What changed
Recorded public historyThe Cohere wrapper accepted a log-probability request but omitted the values from non-streaming replies. The merged path now returns token…
This is the first recorded public view. Further public updates will appear here.
- Cohere log probabilities now survive a vLLM round trip
The Cohere wrapper accepted a log-probability request but omitted the values from non-streaming replies. The merged path now returns token IDs and probabilities.
First published by OTR: 7 Oct, 18:40 SGTPublished by OTR: 7 Oct 2026, 18:40 SGT
Why the view changed
This is the first recorded view. Future source-backed updates will appear here.
Evidence, memory & review 2 linked sources
Named assets or actors: vLLM, Cohere API, log probabilities
Directional takeaway: N/A · NO ENTRY. This is an operational software signal, not an investible price call.
OTR reasoning: The response adapter carries the generated token metadata into the Cohere-shaped result instead of dropping it.
Counterpoint: Evidence is a focused unit test; streaming replies and a tagged release remain unverified.
Source-grounded facts (paraphrased)
- Time not supplied: The Cohere wrapper accepted a log-probability request but omitted the values from non-streaming replies. The merged path now returns token IDs and probabilities. Source records: source-2-1, source-2-2
Source trail
- Upstream merged commit ↗GitHub upstream projectSource event dated 7 October 2026; exact source-page clock remains on the linked record.
- Upstream pull request ↗GitHub upstream projectSource event dated 7 October 2026; exact source-page clock remains on the linked record.
Review status: PASS WITH DISCLOSED LIMITS
This evidence panel does not assign a score or probability. Any forecast estimate is shown separately with its own source and limitations.