Tag: qwen

  • Qwen3.8-Flash-Next Launches: 125B Multimodal MoE with Multi-Token Prediction

    Qwen3.8-Flash-Next Launches: 125B Multimodal MoE with Multi-Token Prediction

    Published by AICodeNews Editorial Team | August 27, 2026

    In a major open-weights release targeting high-throughput developer inference, Qwen3.8-Flash-Next has officially launched across Hugging Face and Alibaba Cloud, introducing a dynamic sparse architecture with native Multi-Token Prediction (MTP).

    Following twelve days after the release of Qwen 3.8-27B, Qwen 3.8-Flash-Next scales up to 125 billion total parameters while activating only 6 billion parameters per token forward pass, delivering rapid generation speeds exceeding 90 tokens per second on consumer and enterprise GPUs.

    1. Core Architecture of Qwen3.8-Flash-Next

    Under the hood, Qwen 3.8-Flash-Next combines extreme expert sparsity with multi-token speculative heads to minimize memory bandwidth bottlenecks:

    • 6B Active Sparsity: Routes tokens across specialized expert layers, requiring only a fraction of compute per token compared to dense 70B models.
    • Multi-Token Prediction (MTP): Generates multiple candidate tokens per forward pass, nearly doubling decoding throughput in local IDE autocompletion.
    • Native Multimodal Perception: Ingests dense UI screenshots, system architecture diagrams, and complex codebases within a 1M token context window.

    2. Hardware Requirements & Inference Benchmarks

    Precision FormatVRAM FootprintRecommended HardwareSustained Throughput
    FP8 Quantized~32 GB1x RTX 5090 (32GB) or Mac 64GB~85 – 92 tok/s
    INT4 GGUF (Q4_K_M)~18.5 GB1x RTX 3090 / 4090 (24GB)~65 – 75 tok/s
    Uncompressed BF16~68 GB2x A100 (80GB) / Multi-GPU~45 – 55 tok/s

    3. Local Deployment: Running Qwen3.8-Flash-Next on vLLM

    Developers can deploy Qwen3.8-Flash-Next locally for private terminal pair-programming and Model Context Protocol (MCP) servers using standard OpenAI-compatible API configurations:

    # Serve Qwen3.8-Flash-Next with vLLM using multi-token prediction
    vllm serve Qwen/Qwen3.8-Flash-Next-FP8 \
      –tensor-parallel-size 1 \
      –max-model-len 32768 \
      –speculative-model Qwen/Qwen3.8-Flash-Next-MTP \
      –num-speculative-tokens 2 \
      –port 8000

    4. Key Takeaways

    • 125B MoE with 6B Active: Qwen3.8-Flash-Next delivers flagship-grade coding reasoning with the memory latency of a lightweight model.
    • Single 24GB GPU Compatible: Runs in 4-bit GGUF or FP8 on consumer RTX 3090/4090 GPUs.
    • Day-One Tooling Support: Native support across vLLM, SGLang, Ollama, and Cline for autonomous agent workflows.

    Bookmark AICodeNews.com for daily updates on open-source AI models, developer benchmarks, and infrastructure news.

  • Alibaba Releases Qwen3.8-27B on Hugging Face: Apache 2.0 Open Weights for Local GPUs

    Alibaba Releases Qwen3.8-27B on Hugging Face: Apache 2.0 Open Weights for Local GPUs

    Alibaba Cloud’s Qwen Team has officially released Qwen 3.8 27B on Hugging Face under the fully permissive Apache 2.0 open-source license.

    Available immediately on the Hugging Face Qwen Repository, Qwen 3.8 27B delivers dense vision-language and high-performance code generation capabilities engineered specifically to run on consumer hardware without sacrificing benchmark intelligence.

    1. Core Architecture & Hardware Requirements for Qwen 3.8 27B

    Unlike massive cloud-only Mixture-of-Experts clusters, Qwen 3 8 27B strikes an optimal balance between parameter scale and local developer accessibility.

    • Single-GPU Deployment: In 4-bit GGUF or official FP8 precision (Qwen3.8-27B-FP8), Qwen 3.8 27B fits comfortably inside a single 24GB GPU (such as an Nvidia RTX 3090 or 4090) or a modern Apple Silicon Mac with 32GB of unified memory.
    • 55.6 GB BF16 Weights: The official repository provides 18 shards of uncompressed BF16 safetensors alongside official FP8 quantized checkpoints.
    • Permissive Apache 2.0 License: Organizations can deploy Qwen 3.8-27B in commercial SaaS products, private on-prem clusters, and terminal coding agents with zero licensing fees.

    2. Benchmark Performance in Software Engineering Workflows

    Early developer testing across developer communities indicates that Qwen 3.8-27B rivals larger 70B parameter models on code completion, multi-language translation, and function calling.

    • SWE-Bench & HumanEval: Scores within striking distance of proprietary closed endpoints while operating entirely offline on private workstations.
    • Ecosystem Integration: Day-one support across llama.cpp, Ollama, vLLM, SGLang, and Unsloth makes running the model locally effortless.

    3. Key Takeaways

    • Official Hugging Face Release: Qwen 3.8-27B is now live under Apache 2.0.
    • Consumer Hardware Friendly: Runs locally on single 24GB GPUs and 32GB Mac Studios using FP8 and 4-bit quantizations.
    • Zero API Dependency: 100% private, self-hosted coding intelligence with no per-token costs.

    Bookmark AICodeNews.com for daily updates on open-weight AI releases, local model deployment guides, and developer tooling.