The title here buries the lede, IMO. Quoting from Epoch AI in the thread:
> We were restricted from disclosing the partnership until around the time o3 launched [...] Our contract specifically prevented us from disclosing information about the funding source and the fact that OpenAI has data access to much but not all of the dataset.
> Regarding training usage: We acknowledge that OpenAI does have access to a large fraction of FrontierMath problems and solutions, with the exception of a unseen-by-OpenAI hold-out set that enables us to independently verify model capabilities. However, we have a verbal agreement that these materials will not be used in model training.
Here, you can read "large fraction" as meaning "everything but the holdout set" - and my understanding is, they haven't disclosed performance on the holdout set.
The title here buries the lede, IMO. Quoting from Epoch AI in the thread:
> We were restricted from disclosing the partnership until around the time o3 launched [...] Our contract specifically prevented us from disclosing information about the funding source and the fact that OpenAI has data access to much but not all of the dataset.
> Regarding training usage: We acknowledge that OpenAI does have access to a large fraction of FrontierMath problems and solutions, with the exception of a unseen-by-OpenAI hold-out set that enables us to independently verify model capabilities. However, we have a verbal agreement that these materials will not be used in model training.
Here, you can read "large fraction" as meaning "everything but the holdout set" - and my understanding is, they haven't disclosed performance on the holdout set.
> However, we have a verbal agreement that these materials will not be used in model training.
That'll do it... clearly no incentive to do otherwise. There should be some form of academic penalty for this kind of (feigned) naivety.
"The mathematicians creating the problems for FrontierMath were not (actively) communicated to about funding from OpenAI."
That is a bit of an understatement given that they were actively contractualy forbidden from being informed about the fact.