SINGAPORE, July 21, 2026 /PRNewswire/ — Arcfra today announced the release of Neutree 1.1, a Model-as-a-Service platform for enterprise AI inference. The new version adds native GPU virtualization and expanded model governance capabilities, helping enterprises improve GPU utilization while managing model usage, quotas, access control, and security audits across AI applications.
As enterprise AI moves from pilots to production, organizations are running more diverse model workloads, including LLMs, OCR, ASR, embedding, rerank, and other task-specific models. GPU passthrough remains important for high-performance LLM inference, but lightweight and high-concurrency small models can leave GPU memory and compute capacity underused when each workload occupies a full GPU.
Neutree 1.1 adds native vGPU support on top of existing GPU passthrough and logical isolation capabilities. Users can enable GPU virtualization as needed and split GPU resources by memory and compute capacity, allowing one GPU to run multiple model instances for workloads such as OCR, speech, embedding, rerank, and multi-service concurrent inference. For performance-sensitive LLM workloads, users can continue using GPU passthrough to access full-card resources.
The release also provides a global view of node-level GPU usage, helping administrators allocate, schedule, and split resources based on actual demand. With hard-isolated resource partitioning, multiple model instances can run on the same GPU without interfering with each other, improving utilization while maintaining predictable service behavior.
Neutree 1.1 also extends its model gateway with a unified governance layer for internal and external models. As model services are connected to more business systems, platform teams need centralized visibility into usage, quotas, access permissions, and request-level traceability. Neutree helps teams manage token usage, request activity, API key quotas, rate limits, concurrency limits, model access scopes, access logs, and security audits without changing existing application calling patterns.
Neutree is a key component of Arcfra’s agentic AI infrastructure for AI inference. Together with Arcfra Enterprise Cloud Platform, it helps enterprises build production-grade, high-performance, and governable model inference infrastructure across compute, storage, Kubernetes, and observability layers.
Foxconn, the world’s largest electronics manufacturer, was among the first customers to try Neutree. By using its existing Arcfra infrastructure as a production-grade AI foundation, Foxconn adopted Neutree to streamline model delivery and unify compute and model resource management.
Neutree 1.1 is now open source on GitHub. For Neutree Enterprise, contact Arcfra.
Source from: Read the full article
Leave a comment