entertainment

Run a 1T parameter model on a 32gb Mac by streaming tensors from NVMe

Lau Chi Fung

— 1 min read

The user requests: "use about 4000 words to Summarize the following news article:" Then there's a block with some ascii art and partial text:

_ _
| | | |_ _ _ __ _ _ _ __ __ _
| |_| | | | | '_ \| | | | '__/ _` |
| _ | |_| | |_) | |_| | | | (_| |
|_| |_|\__, | .__/ \__,_|_| \__,_|
 |___/|_|
 Run models too big for your Mac's memory
H… [+8623 chars]

It looks truncated. The user likely wanted to paste a long article but only a snippet appears. They want a 4000-word summary, which is extremely long (4k words). They say "Summarize the following news article." But we don't have the full article content. We can only see a short snippet of ascii art and a line "Run models too big for your Mac's memory" and "H… [+8623 chars]". So we can't generate a full summary of the article. We need to ask the user for the full article. The best practice is to ask for clarification or for the rest of the article.

Thus we should respond asking for the full article text, as the snippet is insufficient. Also we must keep a professional tone. Let's do that.I’m happy to help craft a detailed summary, but I’ll need the full text of the article to do so.
The excerpt you shared is just a short snippet, so I can’t accurately summarize the entire piece.

Could you please paste the complete article (or provide a link to it) so I can give you the 4,000‑word summary you’re looking for?

Read more