Skip to content

Conversation

@yangky11
Copy link

Hi,

We evaluated our machine learning prover on miniF2F and found 26 new proofs.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 28, 2023
Copy link

@spolu spolu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@yangky11 yangky11 changed the title 26 New Proofs Found by Machines 33 New Proofs Found by Machines Jun 7, 2023
@yangky11
Copy link
Author

yangky11 commented Jun 8, 2023

Hi,

I have re-run a new version of the model, which discovered 7 additional proofs: 5740c4a

@faabian
Copy link
Contributor

faabian commented Sep 8, 2023

Thank you for sharing the proofs! I'm not sure whether we should continue to add proofs to minif2f though because it will lead to training data contamination with Github being a classical source of pretraining data. I will leave it open for now :)

@yangky11
Copy link
Author

yangky11 commented Sep 8, 2023

I totally understand. Feel free to close this PR as you see appropriate.

@faabian
Copy link
Contributor

faabian commented Sep 10, 2023

Let's leave it open for visibility :)

@Adarsh321123
Copy link

@yangky11 What model did you use?

@ucalyptus2
Copy link

Hi @yangky11 , I have the same question as @Adarsh321123

@faabian
Copy link
Contributor

faabian commented Sep 16, 2024

I think this is the LeanDojo / ReProver paper: https://arxiv.org/abs/2306.15626

@yangky11
Copy link
Author

I think this is the LeanDojo / ReProver paper: https://arxiv.org/abs/2306.15626

Yes, but we no longer maintain the Lean 3 model since people have switched to Lean 4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants