Earlier this week Nilay Patel was working on the show notes for the episode of Decoder I guested on, and he texted me to ask if I could recall the time Steve Jobs sent some random developer a link to an article I wrote about the App Store. He wanted to cite it as an example of Daring Fireball being read, at high levels inside Apple, for a long time. I recalled the whole thing vaguely, as a “holy shit” moment, but not specifically. I hadn’t thought about it in years. But I was sure I could find it in the DF archives.

Turns out, I couldn’t find it, because, it turns out, in a fit of inexplicable modesty and humility, I never linked to it. (From a TechCrunch interview I did at the time, after the saga went somewhat viral: “When asked for his response to Steve’s shout-out, Gruber meekly grinned and said, ‘I just smiled.’”)

Here’s the rough timeline of events. On Thursday 8 April 2010, Apple updated the App Store guidelines to ban the use of Adobe’s then-new Flash-to-iPhone compiler. From my post on the change (which, to some degree, broke the news):

Prior to today’s release of the iPhone OS 4 SDK, section 3.3.1 of the iPhone Developer Program License Agreement read, in its entirety:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs.

In the new version of the iPhone Developer Program License Agreement released by Apple today (and which developers must agree to before downloading the 4.0 SDK beta), section 3.3.1 now reads:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

My reading of this new language is that cross-compilers, such as the Flash-to-iPhone compiler in Adobe’s upcoming Flash Professional CS5 release, are prohibited. This also bans apps compiled using MonoTouch — a tool that compiles C# and .NET apps to the iPhone.

This was enormously controversial at the time, but I also thought largely misunderstood by developers. Later that same day, I published another piece articulating my take on Apple’s reasoning for the change, “Why Apple Changed Section 3.3.1”. From that article:

We’re still in the early days of the transition from the PC era to the mobile era. Right now, Apple is winning. There are other winners right now too — RIM is still growing, and Android has grown a ton in the past year.

The App Store platform could turn into a long-term de facto standard platform. That’s how Microsoft became Microsoft. At a certain point developers wrote apps for Windows because so many users were on Windows and users bought Windows PCs because all the software was being written for Windows. That’s the sort of situation that creates a license to print money.

That seems prescient. (The “license to print money” part — not the “RIM is still growing” part.)

So what Apple does not want is for some other company to establish a de facto standard software platform on top of Cocoa Touch. Not Adobe’s Flash. Not .NET (through MonoTouch). If that were to happen, there’s no lock-in advantage. If, say, a mobile Flash software platform — which encompassed multiple lower-level platforms, running on iPhone, Android, Windows Phone 7, and BlackBerry — were established, that app market would not give people a reason to prefer the iPhone.

And, obviously, such a meta-platform would be out of Apple’s control. Consider a world where some other company’s cross-platform toolkit proved wildly popular. Then Apple releases major new features to iPhone OS, and that other company’s toolkit is slow to adopt them. At that point, it’s the other company that controls when third-party apps can make use of these features.

So from Apple’s perspective, changing the iPhone Developer Program License Agreement to prohibit the use of things like Flash CS5 and MonoTouch to create iPhone apps makes complete sense. I’m not saying you have to like this. I’m not arguing that it’s anything other than ruthless competitiveness. I’m not arguing (up to this point) that it benefits anyone other than Apple itself. I’m just arguing that it makes sense from Apple’s perspective — and it was Apple’s decision to make.

Two days later, on 10 April 2010, developer Greg Slepak emailed Steve Jobs to complain about the decision, citing negative sentiment on Hacker News (much has changed since 2010, but some things have not), writing:

Hi Steve,

Lots of people are pissed off at Apple’s mandate that applications be “originally written” in C/C++/Objective-C. If you go, for example, to the Hacker News homepage right now:

http://news.ycombinator.com/

You’ll see that most of the front page stories about this new restriction, with #1 being: “Steve Jobs Has Just Gone Mad” with (currently) 243 upvotes. The top 5 stories are all negative reactions to the TOS, and there are several others below them as well. Not a single positive reaction, even from John Gruber, your biggest fan.

I love your product, but your SDK TOS are growing on it like an invisible cancer.

Sincerely,
Greg

Jobs wrote back to Slepak (starting a brief exchange of emails):

We think John Gruber’s post is very insightful and not negative:

http://daringfireball.net/2010/04/why_apple_changed_section_331

Steve

Slepak posted the exchange to his blog, Tao Effect, and, well, as Jobs himself might have said, “Boom.” (This was a not infrequent thing at the time, where random users or developers would email Jobs, he’d write back with something pithy, and they’d post the exchange. It was kind of crazy — the most famous CEO in the world, just doing customer service email — and his emails were always sharp.)

So, what would you do if Steve Jobs was quoted in a viral blog post saying, “We think «Your Name Here»’s post is very insightful and not negative”? I decided to just sit there with a smug look on my face for a few days (which, arguably, isn’t all that different from what I do most days) and pretend that it was no big deal. I didn’t link to it or mention it on Daring Fireball, and as far as I can tell, I didn’t even tweet it. As best I can recall, I thought I should just play it cool. I mean of course my article about why Apple changed Section 3.3.1 was right. Why brag? Given that Steve Jobs was reading Daring Fireball, I didn’t want him to read a post from me acting like it was a big deal that he’d recommended a piece I wrote and agreed with it.

That was pretty stupid on my part. Or at least silly. My older perspective, today, is not to overthink such things. If something cool happens, I link to it. It seems ridiculous in hindsight that I didn’t link to Slepak’s post. And, I was thinking this week, if I couldn’t find a link to the overall story because I wrongly presumed I must have linked to it at the time, I wondered how many other readers, over the years, have gone hunting for that “very insightful and not negative” story and couldn’t find it because it was never mentioned or linked to on Daring Fireball.

So, today, I wrote the post I should have written back then, and backdated it to 11 April 2010.

To complete the timeline, April 2010 was a busy month. That same month saw HP buy Palm (in a last-ditch effort to remain relevant as the industry rapidly shifted from being PC-centric to mobile-centric), Apple acquire a company called “Siri”, and Gizmodo publish details on the iPhone 4 prototype some poor Apple engineer accidentally left in a bar. The original iPad had just shipped. And at the end of the month, Jobs published “Thoughts on Flash” on the Apple.com homepage. It’s kind of wild that was all in one month — scrolling down the monthly archive page for April 2010 is just one gem after another.

Re-reading “Thoughts on Flash” again now, for the umpteenth time, I’ll say this: I think Steve Jobs’s post was very insightful and not negative.

I’m linking here to Techmeme’s roundup of news coverage and commentary, but I highly recommend you start by reading Gonzalez Rogers’s 80-page decision. It is excoriating. I’ve read few legal decisions quite like it. But it’s also incredibly cogent and plainly written.

From the start:

To summarize: One, after trial, the Court found that Apple’s 30 percent commission “allowed it to reap supracompetitive operating margins” and was not tied to the value of its intellectual property, and thus, was anticompetitive. Apple’s response: charge a 27 percent commission (again tied to nothing) on off-app purchases, where it had previously charged nothing, and extend the commission for a period of seven days after the consumer linked-out of the app. Apple’s goal: maintain its anticompetitive revenue stream. Two, the Court had prohibited Apple from denying developers the ability to communicate with, and direct consumers to, other purchasing mechanisms. Apple’s response: impose new barriers and new requirements to increase friction and increase breakage rates with full page “scare” screens, static URLs, and generic statements. Apple’s goal: to dissuade customer usage of alternative purchase opportunities and maintain its anticompetitive revenue stream. In the end, Apple sought to maintain a revenue stream worth billions in direct defiance of this Court’s Injunction.

There’s quite a bit of fury in those italics. Rule one when you’re in court, any court, is “Don’t piss off the judge.” Apple has absolutely infuriated Gonzales Rogers through actions that all of us saw as outrageous.

In stark contrast to Apple’s initial in-court testimony, contemporaneous business documents reveal that Apple knew exactly what it was doing and at every turn chose the most anticompetitive option. To hide the truth, Vice-President of Finance, Alex Roman, outright lied under oath. Internally, Phillip Schiller had advocated that Apple comply with the Injunction, but Tim Cook ignored Schiller and instead allowed Chief Financial Officer Luca Maestri and his finance team to convince him otherwise. Cook chose poorly. The real evidence, detailed herein, more than meets the clear and convincing standard to find a violation. The Court refers the matter to the United States Attorney for the Northern District of California to investigate whether criminal contempt proceedings are appropriate.

This is an injunction, not a negotiation. There are no do-overs once a party willfully disregards a court order. Time is of the essence. The Court will not tolerate further delays. As previously ordered, Apple will not impede competition. The Court enjoins Apple from implementing its new anticompetitive acts to avoid compliance with the Injunction. Effective immediately Apple will no longer impede developers’ ability to communicate with users nor will they levy or impose a new commission on off-app purchases.

You know a judge is pissed when she busts out the bold italics. Later, on (page 21, citations omitted for readability):

Prior to the June 20 meeting, there were individuals within Apple who were advocating for a commission, and others advocating for no commission. Those advocating for a commission included Mr. Maestri and Mr. Roman. Mr. Schiller disagreed. In an email, Mr. Schiller relayed that, with respect to the proposal for “a 27% commission for 24 hours,” “I have already explained my many issues with the commission concept,” and that “clearly I am not on team commission/fee.” Mr. Schiller testified that, at the time, he “had a question of whether we would be able to charge a commission” under the Injunction, a concern which he communicated.

Schiller comes across as Apple’s sole voice of reason, fairness, and dare I say honesty in this entire ruling. The only people in the world who seemed to think Apple could or should comply with the 2021 injunction (that apps be permitted to steer users to the web to make purchases) by charging a commission — any commission, let alone a 27 percent commission — on those web transactions were Apple’s finance team members, led by Luca Maestri and Alex Roman, and Tim Cook.

Unlike Mr. Maestri and Mr. Roman, Mr. Schiller sat through the entire underlying trial and actually read the entire 180-page decision. That Messrs. Maestri and Roman did neither, does not shield Apple of its knowledge (actual and constructive) of the Court’s findings.

I mean Jesus H. Christ, that’s scathing.

It’s worth pointing out that Luca Maestri is no longer Apple’s CFO. (Kevan Parekh is.) Back in August, Apple announced that Maestri was, effectively retiring as CFO “as part of a planned succession”. Apple’s statement didn’t use the word retiring, but rather the word transitioning. With this ruling and Maestri’s central role in Apple’s decision to forge ahead with a compliance plan where they “allowed” steering to the web by charging the same effective commissions on web transactions as they do for in-app transactions, I now have to wonder whether Maestri retired or “retired”. “Cook chose poorly” (by following Maestri’s recommendation) is not the sort of sentence from a judge that keeps CFOs in their jobs.

As for Alex Roman, I think he’s in some serious trouble. Like doing-time-in-the-clink trouble:

Despite its own considerable evaluation, during the first May 2024 hearing, Apple employees attempted to mislead the Court by testifying that the decision to impose a commission was grounded in AG’s report. The testimony of Mr. Roman, Vice President of Finance, was replete with misdirection and outright lies. He even went so far as to testify that Apple did not look at comparables to estimate the costs of alternative payment solutions that developers would need to procure to facilitate linked-out purchases.

The Court finds that Apple did consider the external costs developers faced when utilizing alternative payment solutions for linked out transactions, which conveniently exceeded the 3% discount Apple ultimately decided to provide by a safe margin. Apple did not rely on a substantiated bottoms-up analysis during its months-long assessment of whether to impose a commission, seemingly justifying its decision after the fact with the AG’s report.

Mr. Roman did not stop there, however. He also testified that up until January 16, 2024, Apple had no idea what fee it would impose on linked-out purchases:

Q. And I take it that Apple decided to impose a 27 percent fee on linked purchases prior to January 16, 2024, correct?

A. The decision was made that day.

Q. It’s your testimony that up until January 16, 2024, Apple had no idea what — what fee it’s going to impose on linked purchases?

A. That is correct.

Another lie under oath: contemporaneous business documents reveal that on the contrary, the main components of Apple’s plan, including the 27% commission, were determined in July 2023.

Neither Apple, nor its counsel, corrected the, now obvious, lies. They did not seek to withdraw the testimony or to have it stricken (although Apple did request that the Court strike other testimony). Thus, Apple will be held to have adopted the lies and misrepresentations to this Court.

There’s so much more. The whole ruling is compelling — and damning — reading.

Keep in mind this whole thing stems from an injunction from a lawsuit filed by Epic Games that Apple largely won. The result of that lawsuit was basically, “OK, Apple wins, Epic loses, but this whole thing where apps in the App Store aren’t allowed to inform users of offers available outside the App Store, or send them to such offers on the web (outside the app) via easily tappable links, is bullshit and needs to stop. If the App Store is not anticompetitive it should be able to compete with links to the web and offers from outside the App Store.”

Are the results of this disastrous for Apple’s App Store business? I don’t think so at all. Gonzales Rogers is demanding that Apple ... do what Phil Schiller recommended they do all along, which is to compete fair and square with purchases available on the web. She’s not demanding they do what, say, Tim Sweeney wanted them to do. She’s basically saying Phil Schiller was right. Read her entire ruling and it’s hard to imagine anyone disagreeing with that.

But are the results of this disastrous for Apple’s reputation and credibility? It sure seems like it. But it would be worse — much worse — for Apple’s reputation if Phil Schiller weren’t still there. Without him, this ruling makes it sound like they’d be lost, both ethically and legally.

I’ll give the final words to Gonzales Rogers’s own closing:

Apple willfully chose not to comply with this Court’s Injunction. It did so with the express intent to create new anticompetitive barriers which would, by design and in effect, maintain a valued revenue stream; a revenue stream previously found to be anticompetitive. That it thought this Court would tolerate such insubordination was a gross miscalculation. As always, the cover-up made it worse. For this Court, there is no second bite at the apple.

It Is So Ordered.

My number one tip for becoming a Mac power user is to get into Keyboard Maestro. Using Keyboard Maestro feels like gaining superpowers. I keep meaning to write more about Keyboard Maestro, and so I’m just going to start documenting all the little use cases I find for it. Here’s one from today.

I use MarsEdit to publish at least 99 percent of the posts on this site. (The other 1 percent are posts I create on my phone, using the web interface for Movable Type.) I use MarsEdit a lot. About once a week or so, I accidentally try to paste text in MarsEdit when I think I have text on my clipboard, but it’s actually an image. When you paste an image in MarsEdit, it’s not like pasting into Mail or Notes or TextEdit, where the image just goes into the text. So MarsEdit, trying to be helpful, opens its Upload Utility window — which, if I were using WordPress or some other CMS, might allow to me upload the image to my server for referencing from the HTML of the blog post. That’s not how my system works, and not how I want it to work, so every time this happens I have to close the Upload Utility window. And every time, I try to do this by hitting the Esc key on my keyboard. But the Upload Utility window isn’t a dialog box with a Cancel button that would be triggered by Esc. It’s a regular window. So after hitting the Esc key, which doesn’t do anything in this context, I then remember, once again, that I need to hit ⌘W instead. (I think I don’t naturally think to hit ⌘W because my instincts tell me ⌘W would try to close the blog window I’m writing in.)

Today it happened again, and finally the notion occurred to me that I could fix this with Keyboard Maestro. My first thought was that I could create a macro that would close the frontmost window in MarsEdit if, and only if, the frontmost window was named “Upload Utility”. A second later it occurred to me that I could probably do better than that, and prevent the Upload Utility window from opening in the first place if I ever try to paste an image in MarsEdit.

I was right. This wasn’t just super easy to create in Keyboard Maestro, it was super quick. I’ve spent 10× more time writing about this macro here than I did creating it. I think that’s why I so seldom write about my little hacks in Keyboard Maestro — they not only save me time and eliminate annoyances once they’re created, but they’re so easy to create that I just get back to whatever I was previously doing after making a new one.

First, I have a group (think: folders) in Keyboard Maestro for every app for which I’ve created app-specific macros. You just create a new group and set it to only be available when one (or more) specific applications are active. Inside my group for MarsEdit, I created a new macro named “Don’t Paste Images”.

It’s triggered by the hot key sequence ⌘V. That means every single time I paste in MarsEdit, this macro will run. Keyboard Maestro is so frigging fast that I’ll never notice. (Keyboard Maestro macros execute so fast that in some scenarios, you have to add steps to pause for, say, 0.2 seconds to keep the macro from getting ahead of the user interface it’s manipulating.)

The macro executes a simple if-then-else action with the following pseudocode logic:

if the System Clipboard has an image
    play a sound
else
    simulate the keystroke ⌘V

That’s the whole thing. And it worked perfectly the first time I tried it. Here’s a screenshot of my macro.

So if I type ⌘V in MarsEdit, and the clipboard contains an image, I just hear a beep. (I could just default to the system beep, but I chose the standard MacOS “Bottle” sound just for this macro — I sort of want to know that it’s this macro keeping me from pasting whatever text I wrongly thought was on my clipboard, so I want a distinctive sound to play.) Nothing gets pasted, so MarsEdit’s Upload Utility window doesn’t appear.

If the clipboard doesn’t contain an image, then Keyboard Maestro simulates a ⌘V shortcut and that gets passed to MarsEdit, and from my perspective as a user, it’s just like a normal paste of the text I expected. I have a few macros that work like this, where the macro is trigged by an application’s own keyboard shortcut, and the macro will (if certain conditions are met) pass through the same simulated keyboard shortcut to the application. When I first tried this, many years ago, I was half worried that it would trigger an infinite loop, where the simulated keystroke from the Keyboard Maestro macro would re-trigger the macro. I was wrong to worry — Keyboard Maestro is too clever for that.

You almost certainly don’t have my particular problem with the occasional inadvertent pasting of images into MarsEdit. But I bet you have your own esoteric annoyances related to your own most-used apps and most-frequent tasks. Keyboard Maestro lets you effectively add your own little features to your favorite apps — often with no “scripting” at all. The best part is, while writing this very blog post, my new “Don’t Paste Images” macro saved me from seeing that cursed Upload Utility window once more, because I had the screenshot of the macro on my clipboard, when I thought I had copied the URL for it on my server.

Extraordinary illustrated essay by Marcin Wichary, documenting a typeface — and its long, fascinating, splintered history — that exemplifies the difference between beautiful and pretty. The beauty in Gorton isn’t just in its plainness and hardworking mechanical roots — it’s in the history of the 20th century itself. Gorton became such a part of the world that the bygone world of the previous century imbues how this font makes me feel.

Do yourself a favor and read this one in a comfortable chair, with a tasty slow-sipping beverage, on a screen bigger than a phone. Everything about this piece is exemplary and astounding — the writing, the photography, the depths of research. But most of all, Wichary’s clear passion and appreciation. It’s a love letter.

A few days ago Nokia unveiled their Design Archive at Aalto University in Finland. Fahad X spotted a real gem — an internal confidential slide deck shared within the company the day after Apple had introduced the original iPhone at Macworld Expo in January 2007. To the credit of the team that put this presentation together, they mostly got it:

• “iPhone touch screen UI may set a new standard of state-of-art: The new user interface may change the standards of the superior user experience for the whole market”. They quote Avi Greengart (a keen and sharp observer): “visually stunning and incredibly responsive”.

• The on-screen keyboard is mentioned only in passing on a deck titled “Other Great Innovations” as “Predictive, corrective input from on-screen qwerty-keypad”. Most of the existing smartphone makers simply could not believe the iPhone’s touchscreen keyboard would work. Steve Ballmer famously laughed his way into getting sacked from Microsoft over the iPhone’s price and lack of a hardware keyboard. BlackBerry was obsessed with hardware keyboards. This Nokia deck is remarkably open to the idea that Apple was onto something. It does have a bullet point, under “iPhone has the biggest impact on the definition of coolness” (true!), that states: “Even though Steve Jobs emphasized iPhone superiority to ‘Buttons’, it is to be expected that the Consumer QWERTY category will continue to succeed.” But still, this deck is remarkable for acknowledging the potential significance of the iPhone’s keyboard.

• It’s a swing and a miss regarding third party software: “No mention either of Java support, unusual user input method may be the reason. Lack of Java would shut out a big mass of existing SW.” So close to getting it right. The iPhone’s lack of support for the then-“dominant” Java ME (Micro Edition) platform did shut out all existing mobile software. But all of that software sucked, big time. Sucked to develop, sucked to distribute, sucked to install, sucked to use. Not supporting it was a huge win for the iPhone, just like not supporting Flash Player wound up being a huge win for both the iPhone in particular and the mobile web generally.

• It’s weird in hindsight that the deck makes mention of Java mobile apps but the only mention of the web is in passing: “Browsing - Safari web browser – (reportedly most advanced ever) Full screen with touch zoom-in functionality.” This speaks to Nokia’s pre-iPhone mindset that mobile platforms were not really internet-first devices but instead were mostly for running shitty software from carriers (for consumers) or corporate IT departments. The iPhone’s excellent day-one version of Mobile Safari, with support for the “real internet”, not the “baby internet” (to use Steve Jobs’s own terms). It’s so easy to get blinded by the way things currently work and to assume they’ll keep working that way.

• The Nokia team nailed the remarkable and potentially industry-changing nature of Apple’s relationship with Cingular:

  • Cingular got multi-year exclusivity to iPhone. In exchange it gave up to Apple in many respects:
    • Cingular has allowed Apple to launch a device with WLAN and inbuilt services
    • The Yahoo and Google experiences are built in to the Apple phone
    • iTunes synch is done in cradle, no OTA
    • The multi-year exclusivity of the Cingular — Apple arrangement makes one ask if Apple gets a share of data revenues or Cingular iTunes revenues?
    • This may be a start for the whole market to change
  • iPhone will be sold both in Apple stores and Cingular stores
  • This is a marriage of convenience built on mutual distrust and recognition that each has assets the other needs

  Nokia concluded, “Other US operators need desperately something against Cingular and Apple” and advised that Nokia should “work very closely especially with T-Mobile.”

The most prescient line in the whole deck was this, recognizing that the iPhone was a UI breakthrough and that the entirety of Nokia’s offerings were far behind:

Nokia needs to develop the touch UI to fight back. S60 should be focus. Maemo platform is critical strength due to openness. Nokia needs a Chief UI architect to re-energize Nokia’s UI innovation across platforms and businesses.

I’d translate that as the presentation team imploring Nokia’s leadership to recognize that Apple’s primary priority was creating and delivering a great experience, both hardware and software, and Nokia had no such institutional value placed on design quality — on coolness — and would need to reinvent itself to prioritize the same ideals if it was going to compete.

Upton Sinclair’s famed adage is apt, as ever: “It is difficult to get a man to understand something, when his salary depends on his not understanding it.”

I thought we were done for major model releases in 2024, but apparently not: Alibaba's Qwen team just dropped the Apache 2.0 licensed Qwen licensed (the license changed) QvQ-72B-Preview, "an experimental research model focusing on enhancing visual reasoning capabilities".

Their blog post is titled QvQ: To See the World with Wisdom - similar flowery language to their QwQ announcement QwQ: Reflect Deeply on the Boundaries of the Unknown a few weeks ago in November.

It's a vision-focused follow-up to QwQ, which I wrote about previousy. QwQ is an impressive openly licensed inference-scaling model: give it a prompt and it will think out loud over many tokens while trying to derive a good answer, similar to OpenAI's o1 and o3 models.

The new QvQ adds vision to the mix. You can try it out on Hugging Face Spaces - it accepts an image and a single prompt and then streams out a very long response where it thinks through the problem you have posed it. There's no option to send a follow-up prompt.

I've tried it out with a bunch of things, with mixed results - but it's really fun seeing how it works through a problem.

My most successful prompt was "Count the pelicans" with this image:

Here's the full response. Some highlights:

So I've got this picture of some pelicans, and I need to count them. Let me see... First off, I know that pelicans are those big birds with the massive beaks, right? [...]

Let me try counting again, more carefully this time. Starting from the left:

1. The first pelican is fully visible, standing at the front.
2. Behind it, slightly to the right, is the second pelican.
3. Then, further back, there's a third pelican.
4. And next to that one, there's a fourth pelican.

So, that's four pelicans in total. But, I notice that on the far right of the image, there's part of another bird, maybe a pelican's back or tail feathers, but it's not fully in the frame, so I don't think I should count it as a whole pelican.

Something about the tone of voice it uses, "First off, I know that pelicans are those big birds with the massive beaks, right?", tickles me.

I also tried this on a much harder image - it counted 17 which looks about right to me but honestly I have trouble with that image myself.

I fed in one of the ARC-AGI puzzles that o3 had failed at.

It produced a very lengthy chain of thought that was almost entirely incorrect, but had some interesting "ideas" in it:

[...] Let me try to think in terms of cellular automata rules, like Conway's Game of Life.

In Game of Life, each cell's state in the next generation is determined by its current state and the states of its eight neighbors.

Perhaps a similar rule applies here.

Finally, I asked it to "Estimate the height of the dinosaur" against this image (which, as it correctly noted, is actually an inflatable dragon):

I'm not sure this task is even possible given the lack of items in the frame of comparable known height. It had a good go at it though! I particularly enjoyed how it began:

So I've got this inflatable Christmas decoration, and it's a dragon! But not just any dragon—it's a green dragon wearing a Santa hat and a red and white scarf. And get this, there's a reindeer on its back, also wearing a Santa hat and holding onto the dragon's wings. It's pretty festive and colorful, with lots of vibrant greens, reds, and yellows.

The dragon has these big, friendly eyes and a smiling face, which makes it look really cheerful. Its wings are spread out, and it has white claws on its feet. The reindeer is brown with antlers and seems to be enjoying the ride on the dragon's back.

Now, the question is, how tall is this inflatable dragon? I need to estimate its height. Let me see. I'm standing next to it, and I'm about 5 feet 10 inches tall. The dragon is definitely taller than me. Maybe around 8 or 9 feet high? But I should try to be more precise.

I wonder how it decided that its own height was 5 feet 10 inches!

Running QvQ locally

All of my experiments so far have used the hosted Hugging Face demo. I'm hoping to try it out on my own laptop soon - I think it should just fit in 64GB of M2, maybe with a smaller quantization.

Right now the options are:

• Qwen/QVQ-72B-Preview on Hugging Face has the GPU model weights, for use with Hugging Face Transformers and the qwen-vl-utils Python package.
• Prince Canuma converted the model for Apple's MLX framework as mlx-community/qvq-72b-preview and got it working with his excellent mlx-vlm package.

As a happy user of Ollama's qwq port I'm hoping they add a QvQ release at some point soon as well.

Running QvQ on my Mac using mlx-vlm

Update: I successfully ran QvQ on my 64GB M2 MacBook Pro using Prince Canuma's mlx-vlm via uv run with the following command:

uv run --with 'numpy<2.0' --with mlx-vlm python \
  -m mlx_vlm.generate \
    --model mlx-community/QVQ-72B-Preview-4bit \
    --max-tokens 10000 \
    --temp 0.0 \
    --prompt "describe this" \
    --image pelicans-on-bicycles-veo2.jpg

My image input here was taken from my experiment with Google's Veo 2 video generation model:

Here's the full output from running that command, after it first downloaded 38GB to ~/.cache/huggingface/hub/models--mlx-community--QVQ-72B-Preview-4bi:

==========
Image: ['pelicans-on-bicycles-veo2.jpg'] 

Prompt: <|im_start|>system
You are a helpful and harmless assistant. You are Qwen developed by Alibaba. You should think step-by-step.<|im_end|>
<|im_start|>user
describe this<|vision_start|><|image_pad|><|vision_end|><|im_end|>
<|im_start|>assistant

Alright, I've got this "Text to Video" tool to explore. It seems pretty advanced, allowing me to turn text descriptions into actual videos. The interface looks user-friendly, with a dark theme that's easy on the eyes. On the left side, there's a panel where I can input my text prompt. It already has an example filled in: "A pelican riding a bicycle along a coastal path overlooking a harbor."

That sounds interesting! I can see four different video thumbnails on the right side, each showing variations of this scene. One of them even has the pelican wearing a helmet, which adds a humorous touch. Another shows the pelican standing next to a bicycle, almost as if it's ready to hop on and start pedaling.

Below the text input, there are some style options like "photorealistic," "35mm film," "minimal," "sketchy," "handmade," and "abstract." These probably let me choose the visual style of the video. For now, I'll stick with the default settings.

At the bottom of the left panel, there are two buttons: "Start over" and "Create videos." I guess "Create videos" is the one I need to press to generate the videos based on my text prompt.

Let me think about this. I want to see what kind of videos this tool can produce based on my description. The example prompt is already filled in, so maybe I should just go ahead and click "Create videos" to see what happens.

Wait a minute, the prompt mentions a coastal path overlooking a harbor. In the thumbnails, I can see a background with water and boats, which matches the description. It's cool how the tool has interpreted the text into visual elements.

I should also pay attention to the "Visual Prompt" section, which says "View only." That probably means I can't edit it directly, but I can view the generated videos based on this prompt.

Hmm, maybe I should try changing the prompt to see different results. For example, instead of a pelican riding a bicycle, maybe I could describe a different scene. But for now, I'll stick with the given example to understand how it works.

After clicking "Create videos," I expect to see the videos being generated. There might be a loading process, depending on the complexity of the prompt and the tool's processing speed.

Once the videos are ready, I can preview them to see how well they match my description. If I'm not satisfied, I can always go back and adjust the prompt or try different style options.

Overall, this seems like a powerful tool for creating custom videos based on textual descriptions. It could be really useful for storytelling, presentations, or even just for fun, experimenting with different scenes and styles.

I think I'll spend some time exploring the different style options to see how they affect the final videos. For instance, selecting "35mm film" might give the videos a vintage, cinematic look, while "abstract" could produce more artistic and less realistic visuals.

Also, I wonder if there's a way to customize more specific aspects, like the camera angle, lighting, or movement. Maybe after generating the initial videos, there are additional editing options available.

In summary, I'm excited to use this Text to Video tool and see what kind of creative possibilities it opens up. With a bit of practice, I could probably create some impressive visual content just by describing what I want to see.

**Final Answer**

To proceed with generating videos based on the provided text prompt, I need to click on the "Create videos" button. This action will initiate the process of converting the textual description into visual content, resulting in multiple video thumbnails that can be previewed for further use or adjustments.
==========
Prompt: 0.870 tokens-per-sec
Generation: 7.694 tokens-per-sec

The license changed from Apache 2.0 to Qwen

When I wrote this post yesterday the LICENSE file in the Hugging Face repository was Apache 2.0. Just after midnight UTC on 25th December this commit landed updating the QVQ-72B-Preview license file to the Qwen license instead.

This looks to me like they were correcting a mistake, not changing their policy. The README.md for that repository has this block of YAML:

license: other
license_name: qwen

And commits to that README at one point linked to the Qwen2.5-72B-Instruct copy of the Qwen license.

The QwQ model repository continues to list Apache 2.0, which matches the YAML in its README as well.

So it looks to me like the intention is for QvQ and Qwen2.5-72B-Instruct to be Qwen licensed, while QwQ is Apache 2.0.

Tags: python, ai, generative-ai, llms, hugging-face, vision-llms, uv, qwen, mlx, inference-scaling