We use essential cookies to keep OpenQiao working. With your permission, optional cookies help us remember your preferences and understand how you use our services. You can accept or reject optional cookies, or manage your choices at any time. Learn more in our Privacy Policy.
openqiao is live: 99.9% SLA · credits never expire · failed requests are not billed Claim 10 CNY credit
🔥 openqiao is live: 10 CNY starting credit, failed requests are not billed Read the docs →
openqiao is live: 99.9% SLA · credits never expire · failed requests are not billed Claim 10 CNY credit
🔥 openqiao is live: 10 CNY starting credit, failed requests are not billed Read the docs →
openqiao is live: 99.9% SLA · credits never expire · failed requests are not billed Claim 10 CNY credit
OpenQiao-Benchmark is a dynamic AI model evaluation leaderboard maintained by the OpenQiao, dedicated to providing comprehensive, objective, and timely performance assessments of AI models. We regularly conduct systematic evaluations of all AI models across every provider channel available on the OpenQiao platform to ensure users have access to the latest and most accurate performance data.
Full cross-channel coverage
We test every available provider channel for each model independently. For example, if the GPT-5 model is available via both OpenAI and Azure, we evaluate each channel separately to reflect differences in performance, stability, and other factors across providers.
Fully open-source and transparent
All test code, procedures, and results are publicly available on GitHub. Anyone can review, verify, and reproduce our findings. Project repository:OpenQiao-Benchmark
We use Scale AI's publicly released dataset,Humanity's Last Exam (Text Only), as our primary evaluation benchmark. This dataset spans a broad range of knowledge domains and reasoning capabilities and is widely recognized in the industry as a high-quality AI evaluation standard.
We run full-scale testing for each model to deliver the most comprehensive performance assessment. However, due to content filtering policies and other technical constraints from certain vendors, some models may be unable to complete all questions. To address this fairly, we apply the following scoring approach:
Scoring: Use the number of questions the model successfully responded to as the total when computing scores.
Cost normalization: Adjust cost proportionally based on the actual completion rate to ensure fairness in cost-effectiveness comparisons.
Transparency: Detailed success rates and completion status for each model can be found in the raw test data for the corresponding provider source in each benchmark release.
OpenQiao-Benchmark aims to build a dynamically updated real-time leaderboard that enables the industry to track the latest performance of AI models. We will continue to refine our methodology, expand evaluation dimensions, and provide users with more comprehensive guidance for model selection.
We welcome community feedback and suggestions! If you have ideas about the evaluation methodology, result analysis, or leaderboard features, please open an issue onOpenQiao-Benchmark-Issue
Thank you for your interest in and support of OpenQiao-Benchmark!
