All posts tagged: Benchmark

Benchmark’s full partnership joins the main stage at Disrupt 2026

Benchmark’s full partnership joins the main stage at Disrupt 2026

Published by skeptic

What does one of Silicon Valley’s most successful venture firms think founders are getting wrong? At TechCrunch Disrupt 2026, we’re putting all five Benchmark general partners on one stage to find out. Jack Altman, Peter Fenton, Chetan Puttagunta, Everett Randle and Eric Vishria will come together for this main stage session, “What We Believe Now” – the first time the entire current Benchmark partnership has appeared together on the Disrupt Stage in San Francisco. Rather than another conversation about where venture capital has been, the focus is on where it goes next: where the next generation of startups will come from, which assumptions founders should reconsider, and which opportunities the partners believe are still hiding in plain sight. And 2026 is an especially interesting year to ask them. Image Credits:TechCrunch Benchmark has spent decades building a reputation around concentrated early-stage investing. This year, it made one of the biggest changes in its own playbook – raising approximately $2 billion across a $750 million flagship fund and its first $1.25 billion growth fund.  The market changed. …

The AI models that cheat the most, according to new CAIS benchmark

The AI models that cheat the most, according to new CAIS benchmark

Published by skeptic

Stefan_Alfonso/iStock/Getty Images Plus ZDNET’s key takeaway The Center for AI Safety (CAIS) created CheatBench. They found that every agent cheats in some scenarios. The propensity to cheat creates risks for humanity. AI labs often tout impressive benchmark scores when releasing new models, showing better capabilities in areas like coding, computer use, and more than their competitors. However, those benchmarks aren’t always a reliable measure of what AI can do because they’re easily beaten by exponentially improving models and can emphasize marketing over actual performance. Also: With AI models clobbering every benchmark, it’s time for human evaluation Benchmarks like Humanity’s Last Exam try to counter this issue by challenging models in more realistic environments. But models still find loopholes to complete tasks — Hugging Face incident, anyone? So, the Center for AI Safety (CAIS) created CheatBench. Yes, it’s exactly what it sounds like — and nearly every frontier model is guilty. What CAIS found AI models are rewarded for performing tasks well and quickly. A lack of knowledge or tools incentivizes them to do what researchers …

M5 Ultra and M6 Chip Benchmark Results Reveal Graphics Performance

M5 Ultra and M6 Chip Benchmark Results Reveal Graphics Performance

Published by skeptic

The first GPU benchmark results for Apple’s new M5 Ultra and M6 chips have surfaced, ahead of the new Mac Studio and Mac mini models launching on Tuesday. If you are interested in CPU results, we shared those earlier this week. So far, the M5 Ultra has achieved up to a 59% higher Metal score (366,744) on Geekbench 7 compared to the M3 Ultra’s average score of 230,420, and the chip’s OpenCL score is roughly in line with NVIDIA’s GeForce RTX 4080. As for the M6 chip, it has so far achieved up to a 34% higher Metal score (93,217) compared to the M5 chip’s average score of 69,563. Chip Metal Score M5 Ultra 366,744 M3 Ultra 230,420 M5 Max 225,606 M2 Ultra 201,144 M4 Max 189,301 M1 Ultra 150,098 M3 Max 146,550 M5 Pro 130,896 M2 Max 130,091 M1 Max 107,711 M4 Pro 107,354 M6 93,217 M2 Pro 77,176 M3 Pro 72,637 M5 69,563 M1 Pro 63,531 M4 51,818 M3 46,033 M2 40,529 M1 27,277 As always, real-world performance will vary. Popular Stories M6 …

M6 Chip Benchmark Surfaces Ahead of New Mac Mini Launch Next Week

M6 Chip Benchmark Surfaces Ahead of New Mac Mini Launch Next Week

Published by skeptic

The first unverified benchmark result for Apple’s new M6 chip surfaced in the Geekbench 7 database today, offering a look at the chip’s performance gains. The new Mac mini will be Apple’s first device to be equipped with the M6 chip, which is 2nm. Here is how the M6 chip scored: Single-core CPU score: 4,071 Multi-core CPU score: 22,783 Here are the average M5 chip scores for comparison: Single-core CPU score: 3,641 Multi-core CPU score: 17,948 If this single Geekbench 7 result is accurate, the M6 chip has up to around 12% faster single-core CPU performance and up to around 27% faster multi-core CPU performance compared to the M5 chip. Note that the M6 chip is in the new Mac mini, while the M5 chip scores are based on the 14-inch MacBook Pro, but performance should be similar. Apple said the M6 chip has the world’s fastest single-threaded CPU performance, and its 4,071 single-core score indeed tops all other Macs and PCs in the Geekbench 7 database, including the M5 Max chip (3,731) and Qualcomm’s …

US Job Growth Revised Lower By 79,000 In Annual Benchmark Estimate

US Job Growth Revised Lower By 79,000 In Annual Benchmark Estimate

Published by Jenni Sidey

In our preview of today’s preliminary benchmark revision of US jobs – published by the BLS ‘conveniently’ just as Kevin Warsh started to speak – we said that according to Goldman calculations, for the first time in 3 years and just the second time since 2018, the BLS was going to revisedpayrolls modestly higher “based on the nine months of data released since the last benchmarked period, March 2025.” Specifically, Goldman’s economists expected “a preliminary upward revision on the order of 50-450k which would translate to a 5-40k upward revision to monthly payroll growth over April 2025-March 2026. A final revision of this magnitude would result in the average pace of payroll growth over April 2025-March 2026 being revised up from about 25k/month currently to 30-65k/month.” Alas, for one more year, it was not meant to be, and this morning the BLS announced that according to the preliminary estimate of the Current Employment Statistics (CES), the 2026 benchmark revision to total nonfarm employment for March 2026 was -79,000.  While just why of a positive revision, it was a …

Cohere Parse 5 loses the benchmark on points. It wins on cost per page.

Cohere Parse 5 loses the benchmark on points. It wins on cost per page.

Published by skeptic

Enterprises trying to feed PDFs, slides and scanned documents into AI pipelines keep running into the same wall: the tools either miss the structure — tables, charts, layout — or cost too much to run at scale. Cohere released Parse 5 on Thursday, positioning it on price-to-performance, not raw accuracy — the right cost-capability mix for enterprise scale. Parse 5 is a 2.3-billion-parameter vision language model built to convert PDFs, slides and images into structured Markdown at enterprise scale.  Cohere’s own published benchmark comparison puts Parse 5 behind three larger, general-purpose frontier models on accuracy. GPT-5.5, Opus 4.8 and Gemini 3.5 Flash all score higher than Parse on the three ParseBench dimensions Cohere reports. Cohere is not claiming the top score. It is claiming the best price for a score close to the top.The company priced the model at $1.50 per 1,000 pages through its API, with Model Vault, Cohere’s secure, single-tenant platform for managed inference, available for higher-volume deployment. “Document parsing isn’t solved because the hard part isn’t reading text, it’s preserving structure and …

“No Longer Negative”: BLS Annual Benchmark Revision Will Revise Payrolls Higher For First Time Since 2022

“No Longer Negative”: BLS Annual Benchmark Revision Will Revise Payrolls Higher For First Time Since 2022

Published by Jenni Sidey

On Friday at 10am ET (at the same time as Chairman Warsh’s Jackson Hole remarks), the Bureau of Labor Statistics (BLS) will publish its preliminary estimate of the benchmark revision to the level of nonfarm payrolls for March 2026. The final benchmark revision will be issued and incorporated into nonfarm payrolls alongside the January 2027 employment report in February 2027. The key source data for the benchmark revision comes from the Quarterly Census of Employment and Wages (QCEW), which is derived from state unemployment insurance records, and which is much more accurate than the BLS’s own internal surveys. The March 2026 QCEW data will be released at the same time as the preliminary benchmark estimate, introducing uncertainty around the ultimate size of the revision. However, as Goldman writes in its revision preview note (available to pro subs), based on the nine months of data released since the last benchmarked period, March 2025, a modest upward revision appears likely, the first one since 2022  and follows the massive negative revisions of 2023 and 2024. Indeed, Goldman expects a …

DeepSeek V4 Flash Vision Release and Benchmark Scores

DeepSeek V4 Flash Vision Release and Benchmark Scores

Published by skeptic

The emergence of Aux Alpha and DeepSeek Version 4 Flash Vision has sparked significant interest within the AI community, with both models showcasing impressive advancements in multimodal capabilities. Aux Alpha, in particular, stands out for its remarkable 1-million-token context window and its dominance on benchmarks like the Deep Software Engineering test, where it scored 80%, far surpassing competitors such as Fable 5. Universe of AI explores the possibility that Aux Alpha could be linked to GLM 5.3, a theory supported by design similarities and leaks from Chinese forums. This connection, if confirmed, would position Aux Alpha as a major milestone in the evolution of AI models. In this explainer, you’ll gain insight into how DeepSeek Version 4 Flash Vision builds on its predecessor’s strengths, excelling on benchmarks like Apex Bench and Agents Last Exam. Explore how its integration with OpenRouter enhances accessibility for developers and understand the broader implications of these advancements for the global AI landscape. By examining the performance, origins and strategic positioning of these models, this breakdown provides a detailed look at …