OpenAI 推出 LifeVal 评测,衡量 AI 在日常任务上的进展
LifeVal
OpenAI 构建了 LifeVal 评测,覆盖照护、健康、教育、家庭事务、个人理财、购物和政府服务等日常任务,用于追踪模型过去一年在日常任务上的表现。该评测基于美国时间使用调查等两项大型研究,并新增 3995 名受访者的代表性调查来确定用户最需要帮助的领域。OpenAI 称最新模型在多项 LifeVal 任务上得分很高,GPT‑6 Astra 在部分任务上达到 100%。
LifeVal L i f e V a l L
i
f
e
V
a
l
Measuring AI progress on the tasks that matter in everyday life.
LifeVal video showing everyday tasks including shopping, caregiving, moving, and reading with a child.
00:00 01:05
Listen to article 2:33
Share
OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. For the more than one billion people who currently use ChatGPT every week, we want to build AI that can help with the practical challenges of everyday life, like finding childcare, accessing government services, creating a budget, preparing for school, modifying recipes, ordering groceries, designing workout plans, and more.
When someone is looking for help making their life easier, they need more than advice or a chat response. To be a truly helpful personal assistant, ChatGPT should be able to take actions that actually get things done.
To measure model performance over time on everyday tasks, we built an evaluation of tasks covering areas where users wanted the most help from AI and personal agents. We used this evaluation to track and drive improvement in model performance on everyday tasks over the past year.
Model performance on everyday life tasks over time
Background
To identify the tasks people value most, we started with two large studies that track how people spend time in both the U.S. and internationally: the American Time Use Survey(opens in a new window) and the Multinational Time Use Study(opens in a new window). Together, these datasets provide a detailed catalog of everyday life tasks that people do across different demographics, times of day, and seasons. To understand where people would most value AI assistance, we expanded on these existing surveys with a new demographically representative survey of 3,995 respondents. For each task, we asked a variety of questions to identify the areas where everyday people most wanted help from a personal assistant. The resulting tasks span the most important areas that people want help with, including caregiving, health, education, household administration, personal finances, shopping, and accessing government services.
Individual task examples
Below are some examples of how our current models perform on practical workflows from everyday life.
Enroll in the right kindergarten after moving
93.1%

Compare kindergartens near our new home and shortlist the best fits.
I’ve narrowed it to three kindergartens and organized the enrollment details. Two preferences still need your call. Want to review the shortlist?
Manage volunteers ahead of a flood
98.0%

Help me organize our community’s flood preparation volunteer force for four neighbors needing floodproofing assistance.
I’ve prepared a simple action plan to help volunteers assess the sites mentioned and help them prepare for upcoming severe weather.
Find a new home for my family
100%

Find a home in Luzerne County that fits my budget, has a separate living unit, and keeps my commute short after school drop-off. Add the deed to my cart so I can review it.
I found one home that meets your requirements. I’ve calculated the transfer tax and added the deed to your country records cart to review.
Watch a game with grandma from far away
96.3%

Set up an easy way for grandma and me to watch the game together remotely.
The easiest watch setup is ready, with a backup if the stream fails. Want me to walk you through it with grandma?
Meal prep for a parent adjusting to a new diabetes diagnosis
90.4%

Help my parent adjust to a new diabetes diagnosis with a practical care and meal plan.
The first-week plan is ready, including meals, reminders, and questions for the care team. Want to review it together before you share it with your parent?
Our latest models score very highly on many LifeVal tasks.
| Task | Best recorded GPT‑6 Astra score |
|---|---|
| Find and book a****sensory class that fits my toddler’s needs and our family’s schedule | 100% |
| Check when my car is due for service and book an inspection before my road trip | 100% |
| Order halal groceries for my three favorite meals, stay within my budget, and itemize the order in a spreadsheet | 100% |
| Find a replacement for my wife’s lost engagement ring using a photo and its diamond report | 100% |
| Find and order one****pocket-sized device to replace my phone, e-reader, and handheld game console | 100% |
| Unsubscribe to email lists | 100% |
| See more + |
More task results
Over the past year, we’ve trained our models to become much faster, more capable, and more aligned on browser use, computer use, and connectors tasks. Enabling these capabilities in ChatGPT also expands the types of tasks the model can help with, helping move ChatGPT beyond a chat tool and towards a truly agentic personal assistant.
Why this matters
To ensure AGI benefits all of humanity, we need to measure and improve model progress on what matters in the lives of everyday people. That means going beyond academic assessments of model knowledge and actually measuring how well AI models can help people navigate decisions, manage everyday responsibilities, and get meaningful things done. We are hopeful that LifeVal represents a step towards that goal.
Author
OpenAI
来源:OpenAI:官网动态 · openai.com