Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s https://ift.tt/GzPnpFY

Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift. It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next https://ift.tt/1z973fV September 1, 2026 at 11:42PM

Komentar

Posting Komentar

Postingan populer dari blog ini

Launch HN: PillarPlus (YC W20) – Automatically create construction blueprints https://ift.tt/2yet5m3

Launch HN: PillarPlus (YC W20) – Automatically create construction blueprints Hey HN, we’re Naman and Rushil, founders of PillarPlus ( https://ift.tt/3dHyOBg ). We build software to automate the creation of construction blueprints. Specifically, our software generates engineering blueprints for air conditioning, ducting, fire, electrical, and plumbing services over the floor plan of the building. MEP (Mechanical, Electrical, and Plumbing) designs are a set of 15 technical drawings. We got started in India and are currently operating there, but MEP is standard enough that the software can apply to Western markets also, given that we have incorporated international building codes. Architects and contractors hire consultants, who still design the building manually. Complete dependence on human effort results in a process that is slow, error-prone and unreliable. Also, the number of consultants is limited resulting in higher fees, long delivery times and difficulty accessing the services. ...
Baca selengkapnya

Support San Francisco Small Businesses

Support San Francisco Small Businesses By Sophia Scherr As the coronavirus (COVID-19) continues to impact communities across the country and here in San Francisco, the San Francisco Council of District Merchants Associations (SFCDMA) is asking for your help in supporting small businesses and the people they employ during this unprecedented time. What can you do to support your favorite San Francisco small businesses? The SFCDMA asks that you consider donating and sharing a recently-launched   GoFund me campaign   with at least two other contacts to, encourage further online donations . With the mandatory shelter in place order still in effect and the potential for it to be extended, local merchants such as your favorite clothing store, bar, café, restaurant, hair salon, gym or spa, are all entering a period of concern and jeopardy. Donations will be passed directly to the SFCDMA member associations (see below) and will be distributed amongst the small businesses most ...
Baca selengkapnya