Z.AI launches open-source GLM-4.6V, a spatial tool for multimodal reasoning calling vision models

Chinese AI startup Zapo AI aka Z.AI has released its GLM-4.6V seriesa new generation of open-source Vision Language Models (VLMS) optimized for multimodal reasoning, front-end automation, and high-performance deployment.

The release includes two models "big" And "small" Size:

1. GLM-4.6V (106b)a large 106 billion parameter model aimed at cloud-scale inference

2. GLM-4.6V-FLASH (9B)a compact model of only 9 billion parameters designed for low-latency, local applications

Note that in general, models with more parameters – or internal settings – or internal settings governing their behavior, i.e. weights and biases – are more powerful, performant and able to perform at a higher general level in more varied tasks.

However, smaller models may offer better performance for edge or real-time applications where latency and resource constraints are critical.

In this regard, the specified innovation is introduced Native function calling In the Vision Language Model—directly implementable tools such as search, crop, or chart recognition with visual inputs.

With a length of 128,000 token contexts (equivalent to 300 pages worth of novel text in a single input/output interaction with the user) and state-of-the-art (SOTA) results in over 20 benchmarks, the GLM-4.6V series is positioned as a highly competitive alternative to both closed and open-source VLMs. It is available in the following forms:

• API access Through an OpenAI-friendly interface

• Try the demo On Zippo’s web interface

• Download the weight With a huggable face

• A desktop assistant app is available Hugging facial spaces

Licensing and Enterprise Use

Distributed under GLM – 4.6V and GLM – 4.6V – Flash MIT Licensea permissive open source license that allows free commercial and non-commercial use, modification, redistribution, and local deployment without liability for open source derivative works.

This licensing model makes the series suitable for enterprise adoption, including scenarios that require complete control over infrastructure, internal governance compliance, or cloud environments.

Model weights and documentation are hosted publicly Hug faceis available with supporting code and tooling GitHub.

The MIT license ensures maximum flexibility for integration into proprietary systems, including internal tools, production pipelines, and edge deployment.

Architecture and technical capabilities

The GLM-4.6V models follow a traditional encoder-decoder architecture with significant adaptations for multimodal input.

Both models incorporate a Vision Transformer (VIT) encoder—based on AIMV2-Hage—and an MLP projector for aligning visual features with a Large Language Model (LLM) decoder.

Video inputs benefit from 3D convolutions and temporal compression, while spatial encoding uses absolute spatial embedding using 2D-rope and bicubic interpolation.

A key technical feature is the system’s support for arbitrary image resolutions and aspect ratios, including wide panoramic inputs up to 200:1.

In addition to static image and document analysis, GLM-4.6V can ingest temporal sequences of video frames with explicit timestamp tokens, enabling robust temporal reasoning.

On the decoding side, the model supports token generation that is coupled with function calling protocols, allowing structured reasoning in text, image, and device outputs. It is supported by supported tokenizer vocabulary and output formatting templates to ensure consistent API or agent compatibility.

Using a native multimodal tool

GLM-4.6V introduces native multimodal function calling, allowing visual assets—such as screenshots, images, and documents—to be passed directly as parameters to tools. This eliminates the need for intermediate text-only conversions, which have historically introduced information loss and complexity.

The tool application procedure works two ways:

• Images or videos can be passed directly to input tools (eg, document pages for cropping or analysis).

• Output tools such as chart renderers or web snapshot utilities return visual data, which is integrated directly into the GLM-4.6V reasoning chain.

In practice, this means the GLM-4.6V can complete tasks such as:

• Generating structured reports from mixed format documents

• Conducting visual audit of candidate photographs

• Automatic data harvesting from documents during generation

• Conducting visual web searches and answering multimodal queries

Higher performance standards than other similar models

The GLM-4.6V was evaluated in over 20 public benchmarks involving general VQA, chart understanding, OCR, STEM reasoning, frontend simulation, and multimodal agents.

According to the benchmark chart released by Zippo AI:

• The GLM-4.6V (106b) achieves SOTA or near SOTA scores among open source models of comparable size (106b) on MMBench, Metvista, MM Longbench, ChartCapro, Refcoco, TreeBench, and more.

• The GLM-4.6V-FLASH (9B) Experiencer outperforms other lightweight models (eg, QWEN3-VL-8B, GLM-4.1V-9B) in almost all categories.

• The 106B model’s 128K-token window allows it to outperform larger models such as the Step 3 (321B) and QWEN3-VL-235B on long-term documentation tasks, video summarization, and structured multimodal reasoning.

Example scores from the leaderboard include:

• Metvista: 88.2 (GLM-4.6V) vs 84.6 (GLM-4.5V) vs 81.4 (QWEN3-VL-8B)

• Web Browser: 81.0 vs 68.4 (QWEN3-VL-8B)

• REF-L4-TEST: 88.9 vs 89.5 (GLM-4.5V), but with better grounding fidelity at 87.7 (flash) vs 86.8

Both models were evaluated using the VLLM inference backend and support SGLANG for video-based tasks.

Front-end automation and long context flows

Zappo AI emphasized the GLM-4.6V’s ability to support front-end development workflows. The model can:

• Generate pixel-accurate HTML/CSS/JS from UI screenshots

• Accept natural language editing commands to edit the layout

• Visually identify and manipulate specific UI components

This capability is integrated into an end-to-end visual programming interface, where the model iterates over layout, design intent, and output code using its native understanding of screen captures.

In long document scenarios, the GLM-4.6V can process up to 128,000 tokens.

• 150 pages of text (input)

• 200 slide deck

• 1 hour videos

Zippo AI reported successful use of the model in financial analysis of multi-document corpora and summarizing full-length sports broadcasts with time-stamped event detection.

Training and reinforcement learning

The model was trained using multistage pre-training followed by supervised fine-tuning (SFT) and reinforcement learning (RL). Key innovations include:

• Curriculum Sampling (RLC): Dynamically adjusts the difficulty of training samples based on the progress of the model

• Multi-domain reward systems: task-specific verifiers for STEM, chart reasoning, GUI agents, video QA, and spatial grounding.

• Function-aware training: structured tags to align reasoning and response formatting (eg, <<<>, <جواب> , <| betar_of_box |>) uses

The reinforcement learning pipeline emphasizes validation rewards (RLVR) over human feedback (RLHF) for scalability, and avoids KL/entropy losses to stabilize training in multimodal domains.

Pricing (API)

Zippo offers competitive pricing for the AIGLM 4.6V series, with the flagship model and its lightweight variant for high accessibility. There is a position.

• GLM-4.6V: $0.30 (input) / $0.90 (output) per 1M tokens

• GLM-4.6V-FLASH: Free

Compared to the large vision-capable and text-first LLM, GLM-4.6V is among the most cost-effective for multimodal reasoning at scale. Below is a comparative snapshot of prices across providers:

USD per 1 meter token – lowest → highest cost set

Previous release: GLM – 4.5 series and enterprise applications

Prior to GLM-4.6V, Z.AI released the GLM-4.5 family in mid-2025, establishing the company as a serious contender in open-source LLM development.

Offering strong performance in standard benchmarks, the flagship GLM – 4.5 and its younger sibling GLM – 4.5 – IAR support both reasoning, tool use, coding and agentive behavior.

The models introduce dual reasoning modes (“thinking” and “non-thinking”) and can automatically generate entire PowerPoint presentations from a single prompt—a feature positioned for use in enterprise reporting, education, and internal comms workflows. Z.AI also extended the GLM-4.5 series with additional variants such as GLM-4.5-X, AIRX, and FLASH, targeting ultra-fast estimation and low-cost scenarios.

Together, these features position the GLM-4.5 series as a cost-effective, open, and production-ready alternative that requires autonomy over model deployment, lifecycle management, and integration pipelines.

Ecosystem Implications

The release of GLM-4.6V represents a significant advance in open source multimodal AI. Although major Vision Language models have proliferated over the past year, some offerings include:

• Using an integrated visual tool

• Structured Multimodal Generation

• Agent-based memory and decision logic

Zappo AI’s emphasis on “closing the loop” is a step toward agent-based multimodal systems through native function calling for process-to-process execution.

The model architecture and training pipeline have shown a continuous evolution of the GLM family, positioning it competitively with offerings such as OpenAI’s GPT-4V and Google DeepMind’s GeminiVL.

Takeaways for enterprise leaders

With GLM-4.6V, Zappo AI introduces an open-source VLM capable of native visualization tools, extended contextual reasoning, and front-end automation. It sets new performance benchmarks in similar-sized models and provides a scalable platform for building agentic, multimodal AI systems..