ai-infrastructure

gm8xx8 @gm8xx8 · 5m THE BLUEPRINT FOR ECOSYSTEM TRUST: - modular components - contributing optimizations upstream - syncing model releases w/ community from Day 0 open infra that scales. TAKE NOTES ↓ > QUOTED: Teortaxes▶️(Deep... @t... · 1h > DeepSeek Open Infra update: DeepSeek announces open-sourcing their inference engine, in collaboration with @lmsysorg SGLang and @vllm_project, by essentially porting it... Show more > [Screenshot of a GitHub README: "open-infra-index / OpenSourcing_DeepSeek_Inference_Engine / README.md", titled "The Path to Open-Sourcing the DeepSeek Inference Engine". Text: "A few weeks ago, during Open Source Week, we open-sourced several libraries. The response from the community has been incredibly positive - sparking inspiring collaborations, productive discussions, and valuable bug fixes. Encouraged by this, we've decided to take another step forward: contributing our internal inference engine back to the open-source community. We are deeply grateful for the open-source ecosystem, without which our progress toward AGI would not be possible. Our training framework relies on PyTorch, and our inference engine is built upon vLLM, both of which have been instrumental in accelerating the training and deployment of DeepSeek models. Given the growing demand for deploying models like DeepSeek-V3 and DeepSeek-R1, we want to give back to the community as much as we can. While we initially considered open-sourcing our full internal inference engine, we identified several challenges: Codebase Divergence: Our engine is based on an early fork of vLLM from over a year ago... Infrastructure Dependencies... Limited Maintenance Bandwidth... Considering these challenges, we've decided to collaborate with the SGLang project and vLLM project as more sustainable alternatives. SGLang and vLLM stand out for their broad hardware support, extensive model compatibility, and state-of-the-art performance... Moving forward, we will work closely with the SGLang team and vLLM team to: Extract Standalone Features... Share Optimizations... We are profoundly grateful for the open-source movement... Note: To clarify, this article outlines our approach to open-sourcing of our DeepSeek-vLLM codebase only. Regarding future model releases, we maintain an open and collaborative stance towards both the open-source community and hardware partners. We commit to proactively synchronizing inference-related engineering efforts prior to new model launches, with the goal of enabling the community to achieve state-of-the-art (SOTA) support from Day-0..."]