Big Data News Hubb
Advertisement
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us
No Result
View All Result
Big Data News Hubb
No Result
View All Result
Home Big Data

Teaching models to express their uncertainty in words

admin by admin
March 26, 2023
in Big Data


We show that a GPT-3 model can learn to express uncertainty about its own answers in natural language—without use of model logits. When given a question, the model generates both an answer and a level of confidence (e.g. “90% confidence” or “high confidence”). These levels map to probabilities that are well calibrated. The model also remains moderately calibrated under distribution shift, and is sensitive to uncertainty in its own answers, rather than imitating human examples. To our knowledge, this is the first time a model has been shown to express calibrated uncertainty about its own answers in natural language. For testing calibration, we introduce the CalibratedMath suite of tasks. We compare the calibration of uncertainty expressed in words (“verbalized probability”) to uncertainty extracted from model logits. Both kinds of uncertainty are capable of generalizing calibration under distribution shift. We also provide evidence that GPT-3’s ability to generalize calibration depends on pre-trained latent representations that correlate with epistemic uncertainty over its answers.



Source link

Previous Post

How We Use Synthetic Data to Improve Performance and Break Away from Dataset Constraints

Next Post

Kobai Rides Atop the Lakehouse with Semantic Graph Engine

Next Post

Kobai Rides Atop the Lakehouse with Semantic Graph Engine

Recommended

Where Tableau Draws the Line on AI Adoption

May 26, 2023

Implement slowly changing dimensions in a data lake using AWS Glue and Delta

March 28, 2023

Field-level security in Amazon OpenSearch Service

November 14, 2022

Don't miss it

News

Why Roblox Picked VictoriaMetrics for Observability Data Overhaul

June 6, 2023
News

Fivetran vs Matillion: Unveiling the Ultimate Battle of ETL Tools

June 5, 2023
Big Data

3 Key AI Predictions for The Near Future + How to Use AI to Transform Your Business

June 5, 2023
Big Data

Better LLMs with Better Data using Cleanlab Studio

June 5, 2023
News

Trakstar unlocks new analytical opportunities for its HR customers with Amazon QuickSight

June 5, 2023
News

Saving Sea Turtles with SAS’s ConserVision App

June 5, 2023
big-data-footer-white

© Big Data News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Big Data
  • News
  • Contact us

Newsletter Sign Up

No Result
View All Result
  • Home
  • Big Data
  • News
  • Contact us

© 2022 Big Data News Hubb All rights reserved.