[{"artifact_id":"9aa6bb1e-170b-41ec-929d-1532858f9390","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-19T22:33:20.7360765Z","update_at":"2026-10-20T08:12:00.602286123Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4.1-Flash-1017","artifact_description":"deepseek-ai/DeepSeek-V4.1-Flash","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/deepseek-v4.1-flash-1017","artifact_template_id":"9aa6bb1e-170b-41ec-929d-1532858f9390","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/deepseek-v4.1-flash-1017","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/DeepSeek-V4.1-Flash-1017\n\nDeepSeek-V4.1-Flash-1017 is an updated version of DeepSeek-V4.1-Flash, delivering substantial improvements in reasoning, coding, agentic tasks, and visual understanding. The model supports a 1M-token context window and is optimized for efficient long-context processing, complex problem-solving, and multi-step agent workflows.\n\nBuilding on DeepSeek-V4.1-Flash, this release improves task execution speed and reduces output length while maintaining strong performance across software engineering and reasoning benchmarks. It supports configurable reasoning modes via the reasoning_effort parameter (low, high, and max), allowing developers to balance reasoning depth, responsiveness, and cost efficiency. DeepSeek-V4.1-Flash-1017 is well suited for coding assistants, chat applications, automated workflows, and agentic systems requiring efficient, high-quality inference.\n\n## API Usage\n\nYou can interact with the deepseek-ai/DeepSeek-V4.1-Flash-1017 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat completions endpoint of DeepSeek-V4.1-Flash-1017.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash-1017\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### Example for Function Calling\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash-1017\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"query_weather\",\n          \"description\": \"Get weather of a city. The user should supply a city first.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"city\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. Beijing\"\n              }\n            },\n            \"required\": [\"city\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"How is the weather in Qingdao today?\"\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\n\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash-1017\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/deepseek-v4.1-flash-1017","cacheHit":false,"pricePer1mPromptToken":180000,"pricePer1mCompletionToken":720000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":4000,"effectiveAt":"2026-10-19T23:09:54.615103Z","endedAt":"","id":"c8866ac9-5b46-43a4-9019-e06163df8653","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":180000,"outputPrice":720000,"cacheReadPrice":3600,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":6000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"api-prod1.gmi-ie.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"69a548d9-2a30-41b4-a3d1-783d8a5f28e4","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_nemotron_3_5_super_vl:llm-engine_v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-01T17:32:27.635597316Z","update_at":"2026-10-27T06:45:30.955502935Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","artifact_description":"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","artifact_tags":["LLM","Text-Image-to-Text"],"artifact_volume_path":"models/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","artifact_template_id":"69a548d9-2a30-41b4-a3d1-783d8a5f28e4","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/Nvidia.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":32768,"quantization":"bf16","info_config_map":[],"provider":"Nvidia","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_p","value":0.95,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"OTEL_SERVICE_NAME","value":"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","is_hidden":true},{"key":"TARGET_ONGOING_REQUESTS","value":"4","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"VLLM_ACTOR_CPUS_PER_GPU","value":"10","is_hidden":true},{"key":"NCCL_NVLS_ENABLE","value":"0","is_hidden":true},{"key":"SERVE_COMMAND","value":"vllm serve /mnt/model_garden --served-model-name nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 --tensor-parallel-size 2 --trust-remote-code --dtype bfloat16 --async-scheduling --gpu-memory-utilization 0.85 --max-model-len 32768 --max-num-seqs 8 --max-num-batched-tokens 8192 --enable-chunked-prefill --limit-mm-per-prompt '{\"video\":1,\"image\":1}' --video-pruning-rate 0.5 --media-io-kwargs '{\"video\":{\"fps\":1,\"num_frames\":128}}' --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser nemotron_v3","is_hidden":true}]},"artifact_details":{"model_description":"# NVIDIA Nemotron 3.5 Super VL\n\nNVIDIA Nemotron 3.5 Super VL is a 120B-parameter vision-language model with 12B active parameters. It combines a hybrid Mamba-2, Mixture-of-Experts, and Attention language backbone with a C-RADIO v4-H vision encoder.\n\nThe model supports text, image, and video inputs with text output. It is designed for document intelligence, GUI understanding, visual reasoning, video question answering, tool use, and multimodal agent workflows.\n\nThis deployment uses the BF16 reference checkpoint. It is primarily intended for evaluation, research, customization, and producing optimized quantized variants.\n\n## API Usage\n\nThe model is available through a RESTful OpenAI-compatible API.\n\nAPI model identifier:\n\n```text\nnvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16\n```\n\n## API Examples\n\n### Generate a Chat Completion\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16\",\n    \"messages\": [\n      {\n        \"role\": \"system\",\n        \"content\": \"You are a knowledgeable AI assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Explain quantum entanglement in simple terms.\"\n      }\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 4096\n  }'\n```\n\n### Disable Reasoning\n\nReasoning is enabled by default. Set `enable_thinking` to `false` for a direct answer.\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16,\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the capital of Japan?\"\n      }\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 4096,\n    \"chat_template_kwargs\": {\n      \"enable_thinking\": false\n    }\n  }'\n```\n\n### Image Understanding\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://example.com/image.jpg\"\n            }\n          },\n          {\n            \"type\": \"text\",\n            \"text\": \"Describe this image in detail.\"\n          }\n        ]\n      }\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 4096,\n    \"chat_template_kwargs\": {\n      \"enable_thinking\": false\n    }\n  }'\n```\n\n### Function Calling\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol, such as AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\",\n    \"temperature\": 0,\n    \"max_tokens\": 1024,\n    \"chat_template_kwargs\": {\n      \"enable_thinking\": true,\n      \"force_nonempty_content\": true\n    }\n  }'\n```"},"model_price":{"modelName":"nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16","cacheHit":false,"pricePer1mPromptToken":100000,"pricePer1mCompletionToken":500000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":100000,"percentageDiscount":0,"effectiveAt":"2026-10-02T05:50:10.572825Z","endedAt":"","id":"57376b31-b0fe-4249-85fa-ea7a821cad5a","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":100000,"outputPrice":500000,"cacheReadPrice":50000,"originInputPrice":100000,"originOutputPrice":500000,"originCacheReadPrice":50000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"538583b4097e.inference-engine-jp4.gmi-serving.com","dashboard_url":"ie-log.gmi-serving.com/user/model-overview?task_id=51919580-c44b-40d0-a274-538583b4097e","num_replicas":1}]},{"artifact_id":"e5aabab2-5ae8-4de1-875b-8aca0d92a833","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-02T20:35:54.139254163Z","update_at":"2026-10-17T09:28:57.737595108Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3.8 Flash","artifact_description":"Gemini 3.8 Flash","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/google/gemini-3.8-flash","artifact_template_id":"e5aabab2-5ae8-4de1-875b-8aca0d92a833","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Google","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.8-flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.8 Flash\n\nGoogle Gemini 3.8 Flash is the most intelligent Flash model in the Gemini 3 series, engineered for long-horizon software engineering, autonomous agents, and complex enterprise workflows—all with the speed and cost efficiency of Flash. It delivers substantial gains over Gemini 3.7 Flash across agentic tasks, multi-step reasoning, and specialized knowledge work, often approaching the performance of higher-cost frontier models at a fraction of the cost.\n\n## API Usage\n\nYou can interact with the Gemini 3.8 Flash model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.8 Flash.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.8-flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.8-flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n#### example for vision (image input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.8-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.8-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n\n#### basic generateContent\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.8-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.8-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.8-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.8-flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```python\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.8-flash:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.8-flash\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.8 Flash reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.8-flash","cacheHit":false,"pricePer1mPromptToken":750000,"pricePer1mCompletionToken":3750000,"originPricePer1mCompletionToken":3750000,"originPricePer1mPromptToken":750000,"percentageDiscount":0,"effectiveAt":"2026-09-02T22:06:27.825043Z","endedAt":"","id":"911417d5-cb9b-4259-8800-36460b25c529","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":750000,"outputPrice":3750000,"cacheReadPrice":75000,"originInputPrice":750000,"originOutputPrice":3750000,"originCacheReadPrice":75000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"60e50c31-b7d0-45a9-b83b-905a0d169c10","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-26T19:07:34.597870963Z","update_at":"2026-10-17T09:28:57.783277846Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-5.3 Flash","artifact_description":"ZAI: GLM-5.3-Flash","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/zai-org/GLM-5.3-Flash","artifact_template_id":"60e50c31-b7d0-45a9-b83b-905a0d169c10","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/kimi.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.3-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# GLM-5.3-Flash\n\nGLM-5.3-Flash is the first natively multimodal model in Z.ai's GLM-5 series. With 320B total parameters and 18B active parameters, it delivers frontier-level coding, agentic, and visual intelligence at roughly one-tenth the cost of GLM-5.2. The model outperforms GLM-5.2 across coding and agentic benchmarks while approaching Claude Opus 4.8 on tasks such as software engineering, tool use, and long-horizon automation.\n\nGLM-5.3-Flash is built for ultra-low-cost inference. It combines a hybrid architecture of sparse and linear attention with Manifold-Constrained Hyper-Connections (mHC), reducing long-context serving cost while preserving precise reasoning up to a 1M-token context window. Trained on a 30T-token multimodal corpus, it supports native image understanding for visual coding, frontend development, document and chart analysis, GUI verification, and agent workflows that require observing rendered output.\n\n## API Usage\n\nYou can interact with the GLM-5.3-Flash model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GLM-5.3-Flash.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for image input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this image?\"},\n          {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/image.jpg\"}}\n        ]\n      }\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of a city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"How is the weather in Qingdao today?\"\n        }\n    ]\n}'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### example for image input\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\"url\": \"https://example.com/image.jpg\"}\n                }\n            ]\n        }\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"zai-org/GLM-5.3-Flash","cacheHit":false,"pricePer1mPromptToken":75000,"pricePer1mCompletionToken":250000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":150000,"percentageDiscount":5000,"effectiveAt":"2026-08-26T19:19:37.957197Z","endedAt":"","id":"ad910b36-0548-427c-9a68-4fbb3657098f","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":75000,"outputPrice":250000,"cacheReadPrice":15000,"originInputPrice":150000,"originOutputPrice":500000,"originCacheReadPrice":30000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"ed878143-66b2-4dbb-a2ee-c05667f8312a","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-20T20:25:28.496308138Z","update_at":"2026-10-17T09:28:57.802476836Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.8 27B","artifact_description":"Qwen/Qwen3.8-27B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.8-27B","artifact_template_id":"ed878143-66b2-4dbb-a2ee-c05667f8312a","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"bf16","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.8-27B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.8 27B\n\nQwen3.8 27B is the compact, deployment-friendly dense model in the Qwen3.8 series — a 27-billion-parameter native vision-language model built for the agent era. It builds upon Qwen3.6-27B with key improvements in coding and office productivity across text and visual modalities, enabling more reliable end-to-end completion of complex tasks. It supports native multimodal input (text, image, video), a 1 million token context window, and stronger long-horizon autonomous execution for coding, professional workflows, and research. It supports hybrid thinking modes (enabled by default), tool calling, MCP integrations, explicit caching, and OpenAI/Anthropic-compatible APIs across agent frameworks (e.g., Claude Code, Qwen Code, OpenClaw). Text-only output.\n\n## API Usage\n\nYou can interact with the Qwen3.8 27B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.8 27B.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3.8-27B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.8-27B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"Qwen/Qwen3.8-27B","cacheHit":false,"pricePer1mPromptToken":450000,"pricePer1mCompletionToken":3200000,"originPricePer1mCompletionToken":3200000,"originPricePer1mPromptToken":450000,"percentageDiscount":0,"effectiveAt":"2026-08-21T04:56:40.744859Z","endedAt":"","id":"5f96c9c1-b64b-4a11-9367-da3498c80a2d","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":450000,"outputPrice":3200000,"cacheReadPrice":250000,"originInputPrice":450000,"originOutputPrice":3200000,"originCacheReadPrice":250000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"824d96ea-74f3-42c1-aa61-c7a21c1e1449","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-19T03:39:40.716955438Z","update_at":"2026-10-17T09:28:57.808889556Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"StepFun: Step-3.7-Flash","artifact_description":"StepFun: Step-3.7-Flash","artifact_tags":["LLM"],"artifact_volume_path":"models/stepfun-ai/Step-3.7-Flash","artifact_template_id":"824d96ea-74f3-42c1-aa61-c7a21c1e1449","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":256000,"quantization":"bf16","info_config_map":[],"provider":"Stepfun"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/stepfun-ai/Step-3.7-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.7,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Step-3.7-Flash\n\nStep 3.7 Flash is a 198B-parameter sparse Mixture-of-Experts (MoE) vision-language model from StepFun. It combines a 196B-parameter language backbone with a 1.8B-parameter vision encoder, activating approximately 11B parameters per token. It supports text and image inputs with text output, a 256k context window, and three selectable reasoning effort levels (`low`, `medium`, `high`) to balance speed and cognitive depth. Built for high-frequency production workloads, it is suited for agentic workflows, tool orchestration, multimodal perception, and coding tasks.\n\n## API Usage\n\nStep-3.7-Flash supports the OpenAI-compatible `/v1/chat/completions` endpoint.\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"stepfun-ai/Step-3.7-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are an AI assistant provided by StepFun.\"},\n      {\"role\": \"user\", \"content\": \"Introduce StepFun'\\''s artificial intelligence capabilities.\"}\n    ],\n    \"reasoning_effort\": \"medium\",\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"stepfun-ai/Step-3.7-Flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain how sparse Mixture-of-Experts improves inference efficiency.\"}\n    ],\n    \"reasoning_effort\": \"medium\",\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### Shell (Image Input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"stepfun-ai/Step-3.7-Flash\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this picture?\"},\n          {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/photo.jpg\"}}\n        ]\n      }\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Shell (Function Call)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"stepfun-ai/Step-3.7-Flash\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"What is the weather in San Francisco?\"}\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_weather\",\n          \"description\": \"Get weather in a location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. San Francisco\"\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n"},"model_price":{"modelName":"stepfun-ai/Step-3.7-Flash","cacheHit":false,"pricePer1mPromptToken":200000,"pricePer1mCompletionToken":1150000,"originPricePer1mCompletionToken":1150000,"originPricePer1mPromptToken":200000,"percentageDiscount":0,"effectiveAt":"2026-08-19T21:00:16.13942Z","endedAt":"","id":"9b50c330-1e6d-4dfc-afdf-d9858975d8b0","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":200000,"outputPrice":1150000,"cacheReadPrice":40000,"originInputPrice":200000,"originOutputPrice":1150000,"originCacheReadPrice":40000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"7e7e7f50-2a84-4ef6-809e-9db5d51bee96","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-13T20:43:24.223296208Z","update_at":"2026-10-17T09:28:57.836143385Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3.7 Flash","artifact_description":"Gemini 3.7 Flash","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemini-3.7-flash","artifact_template_id":"7e7e7f50-2a84-4ef6-809e-9db5d51bee96","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Google","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.7-flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.7 Flash\n\nGoogle Gemini 3.7 Flash is a generally available (GA) model in the Gemini 3 series from Google. It is optimized for coding, agent workflows, software engineering, and knowledge work—including debugging, multi-step planning, and web development. It accepts text, code, images, audio, video, and PDF inputs and produces text output. It offers a 1M token context window for input and up to 65K tokens of output capacity.\n\n\n## API Usage\n\nYou can interact with the Gemini 3.7 Flash model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.7 Flash.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.7-flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.7-flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n#### example for vision (image input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.7-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.7-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n\n#### basic generateContent\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.7-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.7-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.7-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.7-flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```python\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.7-flash:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.7-flash\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.7 Flash reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.7-flash","cacheHit":false,"pricePer1mPromptToken":750000,"pricePer1mCompletionToken":3750000,"originPricePer1mCompletionToken":3750000,"originPricePer1mPromptToken":750000,"percentageDiscount":0,"effectiveAt":"2026-08-13T20:58:11.650614Z","endedAt":"","id":"0be97531-a5f0-4c0d-bf05-81fac2cf56ca","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":750000,"outputPrice":3750000,"cacheReadPrice":75000,"originInputPrice":750000,"originOutputPrice":3750000,"originCacheReadPrice":75000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"bef7e7ab-eeef-413f-8601-e94757f5b91c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_nemotron_3_5_nano_h200:llm-engine_v1.2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-11T21:28:49.421780171Z","update_at":"2026-10-27T06:45:31.043265172Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","artifact_description":"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","artifact_tags":["LLM","Text-Image-to-Text"],"artifact_volume_path":"models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","artifact_template_id":"bef7e7ab-eeef-413f-8601-e94757f5b91c","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/Nvidia.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":65536,"quantization":"bf16","info_config_map":[],"provider":"Nvidia","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_p","value":0.95,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"OTEL_SERVICE_NAME","value":"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVBF16","is_hidden":true},{"key":"TARGET_ONGOING_REQUESTS","value":"4","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"VLLM_ACTOR_CPUS_PER_GPU","value":"10","is_hidden":true},{"key":"SERVE_COMMAND","value":"vllm serve nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 --served-model-name nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 --trust-remote-code --moe-backend flashinfer_cutlass --mamba-backend flashinfer --mamba-cache-mode align --enable-prefix-caching --max-num-batched-tokens 4096 --max-num-seqs 16 --max-model-len 65536 --speculative_config.method mtp --speculative_config.num_speculative_tokens 1 --speculative_config.moe_backend flashinfer_cutlass --reasoning-parser ultra_v3 --reasoning-parser-plugin /mnt/model_garden/ultra_v3_reasoning_parser.py","is_hidden":true}]},"artifact_details":{"model_description":"# Nemotron 3.5 Lightning\n\nNVIDIA Nemotron 3.5 Lightning is the fastest open model in its class for powering always-on agents and completing specialized tasks faster. A 30B MoE model with 3B active\nparameters, it delivers up to 4x higher throughput and 30% faster task completion for\nhigh-volume agentic workflows.\n\nDistilled from NVIDIA’s frontier Nemotron 3 Ultra and developed with the Nemotron\nCoalition, Nemotron 3.5 Lightning is built for popular agent harnesses and designed for\ncustomization. Organizations can post-train it for domain-specific tools, workflows, and\npolicies to achieve leading accuracy on specialized tasks, while keeping control to own,\nadapt, and deploy the model wherever agents run.\n\nAdditionally, NeMo Switchyard can route each agent workflow step to the best available\nmodel from the user’s chosen model pool—helping improve accuracy and efficiency by\nusing models like Nemotron 3.5 Lightning where it is the right fit for high-volume\nspecialized work.\n\n\n## API Usage\n\nYou can interact with the Nemotron 3.5 Lightning model through a RESTful OpenAI-compatible API. The examples below demonstrate how to call the model for text generation, reasoning control, and function calling.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the Nemotron 3 Ultra model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ]\n  }'\n```\n\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\"symbol\"]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```"},"model_price":{"modelName":"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16","cacheHit":false,"pricePer1mPromptToken":150000,"pricePer1mCompletionToken":1000000,"originPricePer1mCompletionToken":1000000,"originPricePer1mPromptToken":150000,"percentageDiscount":0,"effectiveAt":"2026-08-10T04:50:22.014847Z","endedAt":"","id":"86e6cb96-e468-414b-8b22-678617662ee6","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":150000,"outputPrice":1000000,"cacheReadPrice":50000,"originInputPrice":150000,"originOutputPrice":1000000,"originCacheReadPrice":50000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"a9ba47d5a4f7.inference-engine-sg1.gmi-serving.com","dashboard_url":"ie-log.gmi-serving.com/user/model-overview?task_id=e73cc3b6-d3b9-4b37-8867-a9ba47d5a4f7","num_replicas":1}]},{"artifact_id":"8881dcb5-e26b-473c-aa5e-9f43054eecb9","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-03T18:42:50.338304853Z","update_at":"2026-10-17T09:28:57.857166764Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.8 Max","artifact_description":"Qwen/Qwen3.8-Max","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.8-Max","artifact_template_id":"8881dcb5-e26b-473c-aa5e-9f43054eecb9","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.8-Max","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.8 Max\n\nQwen3.8 Max is the largest and most capable model in the Qwen3.8 series — a 2.4-trillion-parameter sparse MoE flagship (95B active per query) built for the agent era. It extends Qwen3.7 Max with native multimodal input (text, image, video), a 1 million token context window, and stronger long-horizon autonomous execution for coding, professional workflows, and research. It supports hybrid thinking modes (enabled by default), tool calling, MCP integrations, explicit caching, and OpenAI/Anthropic-compatible APIs across agent frameworks (e.g., Claude Code, Qwen Code, OpenClaw). Text-only output.\n\n## API Usage\n\nYou can interact with the Qwen3.8 Max model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.8 Max.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3.8-Max\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.8-Max\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"Qwen/Qwen3.8-Max","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":6000000,"originPricePer1mCompletionToken":6000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-08-03T19:20:35.80233Z","endedAt":"","id":"d854f0a3-c7a7-494f-87ad-8dab34f112a2","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":6000000,"cacheReadPrice":250000,"cacheWritePrice":2500000,"originInputPrice":2000000,"originOutputPrice":6000000,"originCacheReadPrice":250000,"originCacheWritePrice":2500000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"5709fa67-fe41-482b-8197-45282e7ea42a","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-31T21:33:00.703130889Z","update_at":"2026-10-17T09:28:57.870412434Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Flash-0731","artifact_description":"deepseek-ai/DeepSeek-V4-Flash-0731","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","artifact_template_id":"5709fa67-fe41-482b-8197-45282e7ea42a","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"DeepSeek","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/DeepSeek-V4-Flash-0731\n\nDeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version. It is an efficiency-optimized Mixture-of-Experts model with 284B total parameters and 13B activated parameters, supporting a 1M-token context window. The model keeps the same architecture and size as the preview release and was re-post-trained with substantially enhanced agentic capabilities.\n\nDeepSeek-V4-Flash-0731 includes hybrid attention (Compressed Sparse Attention and Heavily Compressed Attention) for efficient long-context processing, and supports configurable reasoning modes via the `reasoning_effort` parameter (`low`, `high`, and `max`). It is well suited for applications such as coding assistants, chat systems, and agent workflows where responsiveness, long-context efficiency, and cost efficiency are important.\n\n## API Usage\n\nYou can interact with the deepseek-ai/DeepSeek-V4-Flash-0731 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V4-Flash-0731.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Flash-0731","cacheHit":false,"pricePer1mPromptToken":286000,"pricePer1mCompletionToken":858000,"originPricePer1mCompletionToken":1320000,"originPricePer1mPromptToken":440000,"percentageDiscount":3500,"effectiveAt":"2026-08-29T03:25:48.122941Z","endedAt":"","id":"8e2d13d2-d705-4b8b-9471-8e130a677756","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":286000,"outputPrice":858000,"cacheReadPrice":9100,"originInputPrice":440000,"originOutputPrice":1320000,"originCacheReadPrice":14000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"86199128-2a04-4f63-9028-c4f238172811","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-23T04:31:59.666041685Z","update_at":"2026-10-17T09:28:57.906872502Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3.6 Flash","artifact_description":"Gemini 3.6 Flash","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemini-3.6-flash","artifact_template_id":"86199128-2a04-4f63-9028-c4f238172811","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Google","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.6-flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.6 Flash\n\nGoogle Gemini 3.6 Flash is a generally available (GA) model in the Gemini 3 series from Google. It is optimized for multi-step orchestration, full-stack code refactoring, and general reasoning. It accepts text, code, images, audio, video, and PDF inputs and produces text output. It offers a 1M token context window for input and up to 65K tokens of output capacity.\n\n\n## API Usage\n\nYou can interact with the Gemini 3.6 Flash model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.6 Flash.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.6-flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.6-flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n#### example for vision (image input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.6-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.6-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n\n#### basic generateContent\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.6-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.6-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.6-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.6-flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```python\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.6-flash:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.6-flash\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.6 Flash reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.6-flash","cacheHit":false,"pricePer1mPromptToken":1500000,"pricePer1mCompletionToken":7500000,"originPricePer1mCompletionToken":7500000,"originPricePer1mPromptToken":1500000,"percentageDiscount":0,"effectiveAt":"2026-07-23T04:39:12.030274Z","endedAt":"","id":"2f3a52c5-a7d2-4cd1-a98e-1597b9bdf698","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1500000,"outputPrice":7500000,"cacheReadPrice":150000,"originInputPrice":1500000,"originOutputPrice":7500000,"originCacheReadPrice":150000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"52de98de-cd00-40c8-a99a-5d07576e0b24","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-23T04:14:57.48783873Z","update_at":"2026-10-17T09:28:57.913842092Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3.5 Flash-Lite","artifact_description":"Gemini 3.5 Flash-Lite","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemini-3.5-flash-lite","artifact_template_id":"52de98de-cd00-40c8-a99a-5d07576e0b24","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Google","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.5-flash-lite","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.5 Flash-Lite\n\nGoogle Gemini 3.5 Flash-Lite is a generally available (GA) model in the Gemini 3 series from Google. It delivers fast, cost-effective performance for coding, document understanding, and lightweight agentic workflows. It accepts text, code, images, audio, video, and PDF inputs and produces text output. It offers a 1M token context window for input and up to 65K tokens of output capacity.\n\n\n## API Usage\n\nYou can interact with the Gemini 3.5 Flash-Lite model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.5 Flash-Lite.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.5-flash-lite\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.5-flash-lite\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n#### example for vision (image input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.5-flash-lite\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.5-flash-lite\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n\n#### basic generateContent\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.5-flash-lite:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.5-flash-lite:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.5-flash-lite:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.5-flash-lite\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```python\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.5-flash-lite:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.5-flash-lite\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.5 Flash-Lite reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.5-flash-lite","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":2500000,"originPricePer1mCompletionToken":2500000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2026-07-23T04:38:48.09584Z","endedAt":"","id":"ddae889a-3286-4e86-9165-75a632f7f4f5","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":300000,"outputPrice":2500000,"cacheReadPrice":30000,"originInputPrice":300000,"originOutputPrice":2500000,"originCacheReadPrice":30000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"d36120fa-0025-4018-8daa-54fcc1749135","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-16T21:18:04.557425829Z","update_at":"2026-10-21T18:24:39.610679193Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi: K3","artifact_description":"moonshotai/Kimi: K3","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/moonshotai/kimi-k3","artifact_template_id":"d36120fa-0025-4018-8daa-54fcc1749135","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp4","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Kimi","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/kimi-k3","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Kimi-K3\n\nKimi-K3 is Moonshot AI’s flagship open frontier model, with 2.8 trillion parameters. Built on Kimi Delta Attention (a hybrid linear attention mechanism) and Attention Residuals, it is designed for frontier intelligence across long-horizon coding, knowledge work, and deep reasoning. It features native visual understanding (image and video) and a 1-million-token context window.\n\nThe model supports multimodal inputs and text outputs, with always-on thinking mode for complex reasoning. It is optimized for long-horizon software engineering, agentic workflows, repository-scale coding, tool use, research and knowledge work, and vision-in-the-loop development tasks such as frontend, game, and design iteration.\n\n\n## API Usage\n\nYou can interact with the Kimi-K3 model through a RESTful API using an OpenAI-compatible interface. Kimi-K3 is suitable for coding assistants, agentic development tools, multimodal analysis, research workflows, and automated reasoning systems.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the Kimi-K3 model.\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k3\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful and capable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k3\",\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```\n"},"model_price":{"modelName":"moonshotai/kimi-k3","cacheHit":false,"pricePer1mPromptToken":3000000,"pricePer1mCompletionToken":15000000,"originPricePer1mCompletionToken":15000000,"originPricePer1mPromptToken":3000000,"percentageDiscount":0,"effectiveAt":"2026-07-16T22:00:03.824341Z","endedAt":"","id":"6c89ff9f-efea-46a3-b743-4119a4818eca","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":3000000,"outputPrice":15000000,"cacheReadPrice":300000,"originInputPrice":3000000,"originOutputPrice":15000000,"originCacheReadPrice":300000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"74d835b8-6689-4caa-a580-e4a28128a64d","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-11T01:27:44.081452856Z","update_at":"2026-10-17T09:28:57.941520612Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Grok 4.5","artifact_description":"grok-4.5","artifact_tags":["LLM"],"artifact_volume_path":"models/x-ai/grok-4.5","artifact_template_id":"74d835b8-6689-4caa-a580-e4a28128a64d","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":2000000,"quantization":"fp8","info_config_map":[],"provider":"Grok"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/x-ai/grok-4.5","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# xAI Grok 4.5\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n\nGrok 4.5 is xAI's frontier multimodal model with strong reasoning, coding, and agentic capabilities, and a large context window. Read more about the model on xAI's news post.\n\n## API Usage\n\nYou can interact with the Grok-4.5 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Grok-4.5.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.5\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.5\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.5\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.5\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"x-ai/grok-4.5\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"x-ai/grok-4.5","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":6000000,"originPricePer1mCompletionToken":6000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-08-14T19:54:17.103994Z","endedAt":"","id":"3293576f-6836-478f-8094-04d7ecb2471f","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":6000000,"cacheReadPrice":500000,"originInputPrice":2000000,"originOutputPrice":6000000,"originCacheReadPrice":500000},{"threshold":200000,"inputPrice":4000000,"outputPrice":12000000,"cacheReadPrice":1000000,"originInputPrice":4000000,"originOutputPrice":12000000,"originCacheReadPrice":1000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"16746f80-0f3e-4b41-a812-8627ebbc245b","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-10T05:32:48.738797525Z","update_at":"2026-10-17T09:28:57.948505581Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.6-Luna","artifact_description":"GPT-5.6-Luna","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.6-luna","artifact_template_id":"16746f80-0f3e-4b41-a812-8627ebbc245b","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.6-luna","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI gpt-5.6-luna\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n\nGPT-5.6-Luna is OpenAI’s frontier model designed for complex professional workloads, building on GPT-5.5 with stronger reasoning, higher reliability, and improved token efficiency on hard tasks. It features a 1M+ token context window (922K input, 128K output) with support for text and image inputs, enabling large-scale reasoning, coding, and multimodal workflows within a single system.\n## API Usage\n\nYou can interact with the OpenAI gpt-5.6-luna model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI gpt-5.6-luna.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-luna\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-luna\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-luna\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-luna\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-luna\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n"},"model_price":{"modelName":"openai/gpt-5.6-luna","cacheHit":false,"pricePer1mPromptToken":1000000,"pricePer1mCompletionToken":6000000,"originPricePer1mCompletionToken":6000000,"originPricePer1mPromptToken":1000000,"percentageDiscount":0,"effectiveAt":"2026-07-22T00:51:54.100904Z","endedAt":"","id":"26928bd9-46f8-4776-b580-7437b0b0b0e4","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1000000,"outputPrice":6000000,"cacheReadPrice":100000,"cacheWritePrice":1250000,"originInputPrice":1000000,"originOutputPrice":6000000,"originCacheReadPrice":100000,"originCacheWritePrice":1250000},{"threshold":272000,"inputPrice":2000000,"outputPrice":9000000,"cacheReadPrice":200000,"cacheWritePrice":2500000,"originInputPrice":2000000,"originOutputPrice":9000000,"originCacheReadPrice":200000,"originCacheWritePrice":2500000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"a16c9277-baba-460a-8962-71eeb778ca1d","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-10T05:21:52.933243635Z","update_at":"2026-10-17T09:28:57.954958781Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.6-Terra","artifact_description":"GPT-5.6-Terra","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.6-terra","artifact_template_id":"a16c9277-baba-460a-8962-71eeb778ca1d","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.6-terra","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI gpt-5.6-terra\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n\nGPT-5.6-Terra is OpenAI’s frontier model designed for complex professional workloads, building on GPT-5.5 with stronger reasoning, higher reliability, and improved token efficiency on hard tasks. It features a 1M+ token context window (922K input, 128K output) with support for text and image inputs, enabling large-scale reasoning, coding, and multimodal workflows within a single system.\n## API Usage\n\nYou can interact with the OpenAI gpt-5.6-terra model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI gpt-5.6-terra.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-terra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-terra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-terra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-terra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-terra\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n"},"model_price":{"modelName":"openai/gpt-5.6-terra","cacheHit":false,"pricePer1mPromptToken":2500000,"pricePer1mCompletionToken":15000000,"originPricePer1mCompletionToken":15000000,"originPricePer1mPromptToken":2500000,"percentageDiscount":0,"effectiveAt":"2026-07-22T00:52:39.195442Z","endedAt":"","id":"4fc1b2d1-85f8-42e6-83b2-d1423010c96e","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2500000,"outputPrice":15000000,"cacheReadPrice":250000,"cacheWritePrice":3125000,"originInputPrice":2500000,"originOutputPrice":15000000,"originCacheReadPrice":250000,"originCacheWritePrice":3125000},{"threshold":272000,"inputPrice":5000000,"outputPrice":22500000,"cacheReadPrice":500000,"cacheWritePrice":6250000,"originInputPrice":5000000,"originOutputPrice":22500000,"originCacheReadPrice":500000,"originCacheWritePrice":6250000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"97615ebb-523f-4d09-814d-f022033984f9","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-10T04:33:45.887834885Z","update_at":"2026-10-17T09:28:57.961603081Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.6-Sol","artifact_description":"GPT-5.6-Sol","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.6-sol","artifact_template_id":"97615ebb-523f-4d09-814d-f022033984f9","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.6-sol","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI gpt-5.6-sol\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n\nGPT-5.6-sol is OpenAI’s frontier model designed for complex professional workloads, building on GPT-5.5 with stronger reasoning, higher reliability, and improved token efficiency on hard tasks. It features a 1M+ token context window (922K input, 128K output) with support for text and image inputs, enabling large-scale reasoning, coding, and multimodal workflows within a single system.\n## API Usage\n\nYou can interact with the OpenAI gpt-5.6-sol model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI gpt-5.6-sol.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-sol\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-sol\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-sol\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.6-sol\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.6-sol\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n"},"model_price":{"modelName":"openai/gpt-5.6-sol","cacheHit":false,"pricePer1mPromptToken":5000000,"pricePer1mCompletionToken":30000000,"originPricePer1mCompletionToken":30000000,"originPricePer1mPromptToken":5000000,"percentageDiscount":0,"effectiveAt":"2026-07-22T00:52:24.832985Z","endedAt":"","id":"bc8b6ff6-7bb1-42c3-9183-dd8f0e93814e","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":5000000,"outputPrice":30000000,"cacheReadPrice":500000,"cacheWritePrice":6250000,"originInputPrice":5000000,"originOutputPrice":30000000,"originCacheReadPrice":500000,"originCacheWritePrice":6250000},{"threshold":272000,"inputPrice":10000000,"outputPrice":45000000,"cacheReadPrice":1000000,"cacheWritePrice":12500000,"originInputPrice":10000000,"originOutputPrice":45000000,"originCacheReadPrice":1000000,"originCacheWritePrice":12500000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"c95003e0-4186-45bc-82d6-3270e78cf551","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-06T19:34:53.544587268Z","update_at":"2026-10-17T09:28:57.968850541Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"tencent/Hy3","artifact_description":"tencent/Hy3","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/tencent/Hy3","artifact_template_id":"c95003e0-4186-45bc-82d6-3270e78cf551","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/tencent.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"bf16","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Hunyuan","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/tencent/Hy3","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Tencent Hy3\n\nTencent Hy3 is an open-source Mixture-of-Experts (MoE) large language model developed by Tencent's Hunyuan team. It has 295B total parameters, 21B active parameters, a 3.8B-parameter MTP layer, and a 256K context window. Hy3 is designed for agentic workflows, coding, reasoning, long-context understanding, tool-use scenarios, document analysis, and structured generation.\n\nThe model improves on Hy3 Preview with stronger tool-call reliability, better output-format stability, improved long-context retention, and lower hallucination rates in Tencent's internal evaluations. \n\n## API Usage\n\nYou can interact with the Hy3 model through the proxy's OpenAI-compatible endpoint. `/v1/chat/completions`\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Hy3.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/Hy3\",\n    \"messages\": [\n      {\n        \"role\": \"system\",\n        \"content\": \"You are a helpful and capable AI assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Explain how Mixture-of-Experts models improve inference efficiency.\"\n      }\n    ],\n    \"temperature\": 0.9,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n### Reasoning Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/Hy3\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"Design a high-level migration plan for moving a monolithic Python service to microservices.\"\n      }\n    ],\n    \"temperature\": 0.9,\n    \"top_p\": 1.0,\n    \"max_tokens\": 1200,\n    \"reasoning\": {\n      \"enabled\": true,\n      \"effort\": \"high\"\n    }\n  }'\n```\n\n### Function Call Example\n\nReasoning and tool calling can be used together. Send the same 'reasoning' object alongside OpenAI-compatible 'tools'.\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/Hy3\",\n    \"temperature\": 0.9,\n    \"top_p\": 1.0,\n    \"max_tokens\": 1000,\n    \"reasoning\": {\n      \"enabled\": true,\n      \"effort\": \"high\"\n    },\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```\n"},"model_price":{"modelName":"tencent/Hy3","cacheHit":false,"pricePer1mPromptToken":140000,"pricePer1mCompletionToken":580000,"originPricePer1mCompletionToken":580000,"originPricePer1mPromptToken":140000,"percentageDiscount":0,"effectiveAt":"2026-07-06T23:05:49.837028Z","endedAt":"","id":"9c3f769e-88f3-49e0-bd57-828a5d93c94e","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":140000,"outputPrice":580000,"cacheReadPrice":35000,"originInputPrice":140000,"originOutputPrice":580000,"originCacheReadPrice":35000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"b8c17dda-1188-4c50-90f0-dd2832a251d4","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-06-23T13:12:10.934911487Z","update_at":"2026-10-17T09:28:57.9835997Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi: K2.7-Code-Highspeed","artifact_description":"moonshotai/Kimi: K2.7-Code-Highspeed","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/moonshotai/kimi-k2.7-code-highspeed","artifact_template_id":"b8c17dda-1188-4c50-90f0-dd2832a251d4","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/kimi.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"int4","info_config_map":[],"provider":"Kimi"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/kimi-k2.7-code-highspeed","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Kimi-K2.7-Code \n\nKimi-K2.7-Code is an open-source, code-focused agentic model developed by Moonshot AI. It is designed for advanced software engineering workflows, including long-horizon coding tasks, code generation, debugging, refactoring, repository-level understanding, tool use, and agentic task execution.\n\nThe model supports text-based coding interactions and is optimized for developer productivity, autonomous coding workflows, and complex reasoning over codebases.\n\n\n## API Usage\n\nYou can interact with the Kimi-K2.7-Code model through a RESTful API using an OpenAI-compatible interface. Kimi-K2.7-Code is suitable for coding assistants, agentic development tools, debugging workflows, and automated code review systems.\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the Kimi-K2.7-Code model.\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k2.7-code-highspeed\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful and capable AI coding assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k2.7-code-highspeed\",\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"moonshotai/kimi-k2.7-code-highspeed","cacheHit":false,"pricePer1mPromptToken":1900000,"pricePer1mCompletionToken":8000000,"originPricePer1mCompletionToken":8000000,"originPricePer1mPromptToken":1900000,"percentageDiscount":0,"effectiveAt":"2026-06-23T22:40:52.487353Z","endedAt":"","id":"c93cf94a-2dc5-4240-8863-b5d4fc39d1f2","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1900000,"outputPrice":8000000,"cacheReadPrice":380000,"originInputPrice":1900000,"originOutputPrice":8000000,"originCacheReadPrice":380000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"1f12423b-ac10-4690-a670-36f768d7b9bf","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-06-16T07:32:01.41604057Z","update_at":"2026-10-17T09:28:57.99042427Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"zai-org/GLM-5.2-FP8","artifact_description":"zai-org/GLM-5.2-FP8","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/zai-org/GLM-5.2-FP8","artifact_template_id":"1f12423b-ac10-4690-a670-36f768d7b9bf","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/kimi.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.2-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# GLM-5.2\n\nGLM-5.2 is the latest model in Z.ai's GLM series, continuing the line's focus on coding and long-horizon agentic tasks. Like GLM-5.1, it is designed to plan, execute, and iterate autonomously on extended, engineering-grade tasks. \n\n## API Usage\n\nYou can interact with the GLM-5.2 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GLM-5.2.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.2-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-5.2-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"zai-org/GLM-5.2-FP8","cacheHit":false,"pricePer1mPromptToken":1400000,"pricePer1mCompletionToken":4400000,"originPricePer1mCompletionToken":4400000,"originPricePer1mPromptToken":1400000,"percentageDiscount":0,"effectiveAt":"2026-06-16T07:39:31.936054Z","endedAt":"","id":"7250bbeb-6338-44d4-9cb0-d5118f3aeef2","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1400000,"outputPrice":4400000,"cacheReadPrice":260000,"originInputPrice":1400000,"originOutputPrice":4400000,"originCacheReadPrice":260000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"3c25d31e-d652-4d0b-bb0f-24476766da32","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-06-02T00:38:53.602612211Z","update_at":"2026-10-17T09:28:58.015675699Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMaxAI/MiniMax-M3","artifact_description":"MiniMaxAI/MiniMax-M3","artifact_tags":["LLM"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M3","artifact_template_id":"3c25d31e-d652-4d0b-bb0f-24476766da32","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Minimax","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M3","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# MiniMax-M3\n\nMiniMax-M3 is a multimodal foundation model from MiniMax. It supports text, image, and video inputs with text output, a 1M-token context window, and is suited for long-horizon agentic work, coding, and tool use. It is built on MiniMax Sparse Attention (MSA), which replaces full attention with KV-block selection to cut per-token compute at long context — roughly 1/20 the cost of the previous generation at 1M tokens, with substantially faster prefill and decode while retaining quality across most tasks.\n\nTrained as a native multimodal model on interleaved data and tuned for multi-turn, production-like collaboration via an interactive user-simulator framework, the model is oriented toward sustained, multi-step tasks rather than single-turn execution.\n\n## API Usage\n\nMiniMax-M3 supports two API formats:\n- **OpenAI-compatible**: /v1/chat/completions endpoint\n- **Anthropic-compatible**: /v1/messages endpoint (with extended thinking support)\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M3\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M3\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\", \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Anthropic-Compatible Endpoint (with Extended Thinking)\n\nThe /v1/messages endpoint provides access to the model's reasoning process through the \"thinking\" content block. You can also use the Anthropic SDK directly by configuring the base URL.\n\n#### Using Anthropic SDK (Recommended)\n\n```\n# Configure environment variables\nexport ANTHROPIC_BASE_URL=https://api.minimax.io/anthropic\nexport ANTHROPIC_API_KEY=${YOUR_API_KEY}\n```\n\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nmessage = client.messages.create(\n    model=\"MiniMax-M3\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"text\",\n                    \"text\": \"Hi, how are you?\"\n                }\n            ]\n        }\n    ]\n)\n\nfor block in message.content:\n    if block.type == \"thinking\":\n        print(f\"Thinking:\\n{block.thinking}\\n\")\n    elif block.type == \"text\":\n        print(f\"Text:\\n{block.text}\\n\")\n```\n\n#### Shell (Direct API)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M3\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Solve this step by step: What is 23 * 47?\"}\n    ]\n  }'\n```\n\n#### Shell (with Streaming)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M3\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Tell me a three sentence bedtime story about a unicorn.\"}\n    ],\n    \"stream\": true\n  }'\n```\n\n#### Python (with Streaming)\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nstream = client.messages.create(\n    model=\"MiniMax-M3\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Hi, how are you?\"}]}\n    ],\n    stream=True,\n)\n\nfor chunk in stream:\n    if chunk.type == \"content_block_delta\":\n        if hasattr(chunk, \"delta\") and chunk.delta:\n            if chunk.delta.type == \"thinking_delta\":\n                print(chunk.delta.thinking, end=\"\", flush=True)\n            elif chunk.delta.type == \"text_delta\":\n                print(chunk.delta.text, end=\"\", flush=True)\n```\n"},"model_price":{"modelName":"MiniMaxAI/MiniMax-M3","cacheHit":false,"pricePer1mPromptToken":240000,"pricePer1mCompletionToken":960000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":2000,"effectiveAt":"2026-09-09T00:56:38.312871Z","endedAt":"","id":"722ed9c7-fbe7-4b6d-972c-0776d717bf20","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"claudeV2","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":240000,"outputPrice":960000,"cacheReadPrice":48000,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":60000},{"threshold":512000,"inputPrice":480000,"outputPrice":1920000,"cacheReadPrice":96000,"originInputPrice":600000,"originOutputPrice":2400000,"originCacheReadPrice":120000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"a1f45766-f68a-446d-8bff-513173e8044f","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-05-21T23:15:54.890022631Z","update_at":"2026-10-17T09:28:58.041449898Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.7 Max","artifact_description":"Qwen/Qwen3.7-Max","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.7-Max","artifact_template_id":"a1f45766-f68a-446d-8bff-513173e8044f","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Qwen","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.7-Max","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.7 Max\n\nQwen3.7 Max is the largest and most capable model in the Qwen3.7 series — a next-generation flagship designed for the agent era. Built as a versatile agent foundation, it excels at programming, office and productivity workflows, and sustained long-horizon autonomous execution across hundreds or thousands of steps. It generalizes across agent frameworks (e.g., Claude Code, OpenClaw, Qwen Code), supports hybrid thinking modes (enabled by default), tool calling, MCP integrations, explicit caching, and a 1 million token context window. The base model supports text-only input and output.\n\n## API Usage\n\nYou can interact with the Qwen3.7 Max model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.7 Max.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3.7-Max\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.7-Max\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"Qwen/Qwen3.7-Max","cacheHit":false,"pricePer1mPromptToken":2500000,"pricePer1mCompletionToken":7500000,"originPricePer1mCompletionToken":7500000,"originPricePer1mPromptToken":2500000,"percentageDiscount":0,"effectiveAt":"2026-05-22T00:05:31.862103Z","endedAt":"","id":"4f029a62-faf6-43f0-933d-7f6b6e1ee962","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2500000,"outputPrice":7500000,"cacheReadPrice":250000,"cacheWritePrice":3125000,"originInputPrice":2500000,"originOutputPrice":7500000,"originCacheReadPrice":250000,"originCacheWritePrice":3125000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"88943704-646d-47f6-868c-408ed6f78e08","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-05-19T18:11:50.230957164Z","update_at":"2026-10-17T09:28:58.054471278Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3.5 Flash","artifact_description":"Gemini 3.5 Flash","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemini-3.5-flash","artifact_template_id":"88943704-646d-47f6-868c-408ed6f78e08","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Google"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.5-flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.5 Flash\n\nGoogle Gemini 3.5 Flash is a generally available (GA) model in the Gemini 3 series from Google. It delivers near-Pro intelligence at Flash-tier cost and speed, with strong coding proficiency, parallel agentic execution, and support for long-horizon workflows. It accepts text, code, images, audio, video, and PDF inputs and produces text output. It offers a 1M token context window for input and up to 65K tokens of output capacity.\n\n\n## API Usage\n\nYou can interact with the Gemini 3.5 Flash model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.5 Flash.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.5-flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.5-flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n#### example for vision (image input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.5-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.5-flash\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n\n#### basic generateContent\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.5-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.5-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.5-flash:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.5-flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```python\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.5-flash:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.5-flash\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.5 Flash reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.5-flash","cacheHit":false,"pricePer1mPromptToken":1500000,"pricePer1mCompletionToken":9000000,"originPricePer1mCompletionToken":9000000,"originPricePer1mPromptToken":1500000,"percentageDiscount":0,"effectiveAt":"2026-05-19T20:40:31.774657Z","endedAt":"","id":"cef11d5f-0255-46d1-b2c0-f6f70e161021","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1500000,"outputPrice":9000000,"cacheReadPrice":150000,"cacheWritePrice":83330,"originInputPrice":1500000,"originOutputPrice":9000000,"originCacheReadPrice":150000,"originCacheWritePrice":83330}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"3e23bf2a-2353-4ad5-9e9a-ad04014e329f","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-29T20:02:43.230923126Z","update_at":"2026-10-17T09:28:58.073366267Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiMo-V2.5-Pro","artifact_description":"MiMo-V2.5-Pro","artifact_tags":["LLM"],"artifact_volume_path":"models/XiaomiMiMo/MiMo-V2.5-Pro","artifact_template_id":"3e23bf2a-2353-4ad5-9e9a-ad04014e329f","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/xiaomimimo2.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"bf16","info_config_map":[],"provider":"XiaomiMiMo"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/XiaomiMiMo/MiMo-V2.5-Pro","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Xiaomi MiMo-V2.5-Pro\n##\n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n\nMiMo-V2.5-Pro is Xiaomi’s strongest MiMo model to date, focused on **agentic behavior**, **complex software engineering**, and **long-horizon coherence**. It is a ~1.02T-parameter sparse Mixture-of-Experts model with ~42B active parameters, built on a **hybrid-attention** architecture with a **up to 1M-token** context window (see the [MiMo-V2.5-Pro announcement](https://mimo.xiaomi.com/mimo-v2-5-pro)). In agent harnesses it can sustain very long runs (for example, workloads involving large numbers of tool calls) while following nuanced instructions across ultra-long context. The series is also noted for strong **token efficiency** on agentic evaluations relative to some frontier models at similar capability. Use API model id `mimo-v2.5-pro` on Xiaomi’s platform; on this gateway the routed model name is `XiaomiMiMo/MiMo-V2.5-Pro`.\n\n## API Usage\n\nYou can interact with MiMo-V2.5-Pro through standard OpenAI-compatible HTTP APIs. Below are examples using the chat completions endpoint.\n\n**Message roles:** Examples use OpenAI’s `developer` role where shown; if your vendor rejects it, use `system` instead.\n\n## API Examples\n\nGenerate a model response using the chat completions endpoint of MiMo-V2.5-Pro.\n\n### Create chat completion\n\nThe Chat Completions API generates a model reply from a list of conversation messages.\n\n#### Default\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n```\n\n#### Image Input\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Functions\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"XiaomiMiMo/MiMo-V2.5-Pro\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n"},"model_price":{"modelName":"XiaomiMiMo/MiMo-V2.5-Pro","cacheHit":false,"pricePer1mPromptToken":304500,"pricePer1mCompletionToken":609000,"originPricePer1mCompletionToken":870000,"originPricePer1mPromptToken":435000,"percentageDiscount":3000,"effectiveAt":"2026-07-15T23:19:14.918243Z","endedAt":"","id":"1a448eca-cf11-42fa-9086-dd7dc49653f1","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":304500,"outputPrice":609000,"cacheReadPrice":2520,"originInputPrice":435000,"originOutputPrice":870000,"originCacheReadPrice":3600}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"39c41644-705e-4f94-894f-475b0e3877a9","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-27T03:37:42.331227921Z","update_at":"2026-10-17T09:28:58.086867757Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.5","artifact_description":"GPT-5.5","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.5","artifact_template_id":"39c41644-705e-4f94-894f-475b0e3877a9","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.5","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI gpt-5.5\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n        - [Web search](#web-search)\n\nGPT-5.5 is OpenAI’s frontier model designed for complex professional workloads, building on GPT-5.4 with stronger reasoning, higher reliability, and improved token efficiency on hard tasks. It features a 1M+ token context window (922K input, 128K output) with support for text and image inputs, enabling large-scale reasoning, coding, and multimodal workflows within a single system.\n## API Usage\n\nYou can interact with the OpenAI gpt-5.5 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI gpt-5.5.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(response.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.5\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n#### Web search\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.5\",\n    \"tools\": [{ \"type\": \"web_search_preview\" }],\n    \"input\": \"What was a positive news story from today?\"\n  }'\n```"},"model_price":{"modelName":"openai/gpt-5.5","cacheHit":false,"pricePer1mPromptToken":5000000,"pricePer1mCompletionToken":30000000,"originPricePer1mCompletionToken":30000000,"originPricePer1mPromptToken":5000000,"percentageDiscount":0,"effectiveAt":"2026-04-27T08:41:00Z","endedAt":"","id":"17fb8620-99e9-404a-9b14-98696613f912","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":5000000,"outputPrice":30000000,"cacheReadPrice":500000,"originInputPrice":5000000,"originOutputPrice":30000000,"originCacheReadPrice":500000},{"threshold":272000,"inputPrice":10000000,"outputPrice":45000000,"cacheReadPrice":1000000,"originInputPrice":10000000,"originOutputPrice":45000000,"originCacheReadPrice":1000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"cbecb2f4-480e-4d25-b3d7-8f38ea0c3913","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-24T03:45:06.333061207Z","update_at":"2026-10-17T09:28:58.100819487Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Pro","artifact_description":"deepseek-ai/DeepSeek-V4-Pro","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Pro","artifact_template_id":"cbecb2f4-480e-4d25-b3d7-8f38ea0c3913","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Pro","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/DeepSeek-V4-Pro\n\nDeepSeek-V4-Pro is a powerful large language model maintaining strong performance. \n\nDeepSeek-V4 series incorporate several key upgrades in architecture and optimization:\n\nHybrid Attention Architecture: We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only 27% of single-token inference FLOPs and 10% of KV cache compared with DeepSeek-V3.2.\nManifold-Constrained Hyper-Connections (mHC): We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.\nMuon Optimizer: We employ the Muon optimizer for faster convergence and greater training stability.\n\n## API Usage\n\nYou can interact with the DeepSeek-V4-Pro model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V4-Pro.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Pro","cacheHit":false,"pricePer1mPromptToken":957000,"pricePer1mCompletionToken":1914000,"originPricePer1mCompletionToken":3480000,"originPricePer1mPromptToken":1740000,"percentageDiscount":4500,"effectiveAt":"2026-08-29T04:42:52.555486Z","endedAt":"","id":"ec8e0e46-ed9f-46cb-a7cb-6155b418b7b4","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":957000,"outputPrice":1914000,"cacheReadPrice":79750,"originInputPrice":1740000,"originOutputPrice":3480000,"originCacheReadPrice":145000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"4e4d2e35-34ff-45dc-b09f-c50d464d3464","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-07T09:16:11.594538652Z","update_at":"2026-10-17T09:28:58.139475955Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-5.1","artifact_description":"zai-org/GLM-5.1-FP8","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/zai-org/GLM-5.1-FP8","artifact_template_id":"4e4d2e35-34ff-45dc-b09f-c50d464d3464","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.1-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# GLM-5\n\nGLM-5 is Z.ai’s flagship open-source foundation model engineered for complex systems design and long-horizon agent workflows. Built for expert developers, it delivers production-grade performance on large-scale programming tasks, rivaling leading closed-source models. With advanced agentic planning, deep backend reasoning, and iterative self-correction, GLM-5 moves beyond code generation to full-system construction and autonomous execution.\n\n## API Usage\n\nYou can interact with the GLM-5 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GLM-5.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.1-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-5.1-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"zai-org/GLM-5.1-FP8","cacheHit":false,"pricePer1mPromptToken":1400000,"pricePer1mCompletionToken":4400000,"originPricePer1mCompletionToken":4400000,"originPricePer1mPromptToken":1400000,"percentageDiscount":0,"effectiveAt":"2026-04-22T07:21:37.858745Z","endedAt":"","id":"db02aaa6-b894-425a-9cf5-7daf412813fa","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1400000,"outputPrice":4400000,"cacheReadPrice":260000,"originInputPrice":1400000,"originOutputPrice":4400000,"originCacheReadPrice":260000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"951fef32-e6c8-4a1d-8cac-89405daae6e0","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-02T03:40:32.943573951Z","update_at":"2026-10-17T09:28:58.158036075Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-5","artifact_description":"zai-org/GLM-5-FP8","artifact_tags":["LLM"],"artifact_volume_path":"models/zai-org/GLM-5-FP8","artifact_template_id":"951fef32-e6c8-4a1d-8cac-89405daae6e0","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"fp8","info_config_map":[{"key":"sypports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# GLM-5\n\nGLM-5 is Z.ai’s flagship open-source foundation model engineered for complex systems design and long-horizon agent workflows. Built for expert developers, it delivers production-grade performance on large-scale programming tasks, rivaling leading closed-source models. With advanced agentic planning, deep backend reasoning, and iterative self-correction, GLM-5 moves beyond code generation to full-system construction and autonomous execution.\n\n## API Usage\n\nYou can interact with the GLM-5 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GLM-5.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-5-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"zai-org/GLM-5-FP8","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":1920000,"originPricePer1mCompletionToken":3200000,"originPricePer1mPromptToken":1000000,"percentageDiscount":4000,"effectiveAt":"2026-07-18T08:00:00Z","endedAt":"","id":"774f3f6e-f6ba-4784-b33d-ac8997fda039","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":600000,"outputPrice":1920000,"cacheReadPrice":120000,"originInputPrice":1000000,"originOutputPrice":3200000,"originCacheReadPrice":200000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"a6aeab85-5b61-45b3-bffd-c694b1b232e1","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-23T02:49:14.404060411Z","update_at":"2026-10-17T09:28:58.171851144Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.4-nano","artifact_description":"GPT-5.4-nano","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.4-nano","artifact_template_id":"a6aeab85-5b61-45b3-bffd-c694b1b232e1","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":409600,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.4-nano","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI GPT-5.4-nano\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n        - [Web search](#web-search)\n\nGPT-5.4 nano is the most lightweight and cost-efficient variant of the GPT-5.4 family, optimized for speed-critical and high-volume tasks. It supports text and image inputs and is designed for low-latency use cases such as classification, data extraction, ranking, and sub-agent execution.\n\nThe model prioritizes responsiveness and efficiency over deep reasoning, making it ideal for pipelines that require fast, reliable outputs at scale. GPT-5.4 nano is well suited for background tasks, real-time systems, and distributed agent architectures where minimizing cost and latency is essential.\n\nYou can interact with the OpenAI GPT-5.4-nano model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT-5.4-nano.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-nano\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-nano\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-nano\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(response.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-nano\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n#### Web search\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-nano\",\n    \"tools\": [{ \"type\": \"web_search_preview\" }],\n    \"input\": \"What was a positive news story from today?\"\n  }'\n```"},"model_price":{"modelName":"openai/gpt-5.4-nano","cacheHit":false,"pricePer1mPromptToken":200000,"pricePer1mCompletionToken":1250000,"originPricePer1mCompletionToken":1250000,"originPricePer1mPromptToken":200000,"percentageDiscount":0,"effectiveAt":"2026-03-23T06:24:28.65506Z","endedAt":"","id":"63582e2d-0f83-422b-ba63-6364bfa87b8d","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":200000,"outputPrice":1250000,"cacheReadPrice":20000,"originInputPrice":200000,"originOutputPrice":1250000,"originCacheReadPrice":20000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"23cb6299-525b-4014-bcb0-eeead5563359","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-23T02:49:10.575327584Z","update_at":"2026-10-17T09:28:58.178543194Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.4-mini","artifact_description":"GPT-5.4-mini","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.4-mini","artifact_template_id":"23cb6299-525b-4014-bcb0-eeead5563359","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":409600,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.4-mini","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI GPT-5.4-mini\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n        - [Web search](#web-search)\n\nGPT-5.4 mini brings the core capabilities of GPT-5.4 to a faster, more efficient model optimized for high-throughput workloads. It supports text and image inputs with strong performance across reasoning, coding, and tool use, while reducing latency and cost for large-scale deployments.\n\nThe model is designed for production environments that require a balance of capability and efficiency, making it well suited for chat applications, coding assistants, and agent workflows that operate at scale. GPT-5.4 mini delivers reliable instruction following, solid multi-step reasoning, and consistent performance across diverse tasks with improved cost efficiency.\n## API Usage\n\nYou can interact with the OpenAI GPT-5.4-mini model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT-5.4-mini.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-mini\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-mini\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-mini\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(response.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4-mini\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n#### Web search\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-mini\",\n    \"tools\": [{ \"type\": \"web_search_preview\" }],\n    \"input\": \"What was a positive news story from today?\"\n  }'\n```"},"model_price":{"modelName":"openai/gpt-5.4-mini","cacheHit":false,"pricePer1mPromptToken":750000,"pricePer1mCompletionToken":4500000,"originPricePer1mCompletionToken":4500000,"originPricePer1mPromptToken":750000,"percentageDiscount":0,"effectiveAt":"2026-03-23T06:22:54.877701Z","endedAt":"","id":"225b25fc-978d-4a9b-97b5-b011312af267","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":750000,"outputPrice":4500000,"cacheReadPrice":75000,"originInputPrice":750000,"originOutputPrice":4500000,"originCacheReadPrice":75000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"b55d47ed-0d1e-467e-ae93-f1500402d184","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-18T21:08:02.715781135Z","update_at":"2026-10-17T09:28:58.192996764Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMaxAI/MiniMax-M2.7","artifact_description":"MiniMaxAI/MiniMax-M2.7","artifact_tags":["LLM"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M2.7","artifact_template_id":"b55d47ed-0d1e-467e-ae93-f1500402d184","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":196608,"quantization":"fp8","info_config_map":[],"provider":"Minimax"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M2.7","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# MiniMax-M2.7\n\nMiniMax-M2.7 is MiniMax's first model deeply participating in its own evolution. It delivers outstanding performance in real-world software engineering—including end-to-end full project delivery, log analysis, bug troubleshooting, code security, and machine learning—while also excelling at complex agentic workflows with Agent Teams, dynamic tool search, and persistent memory. On SWE-Pro it scored 56.22%, on VIBE-Pro 55.6%, and on Terminal Bench 2 57.0%. The model maintains a 97% skill adherence rate while working with over 40 complex skills (each exceeding 2,000 tokens), and its GDPval-AA ELO score of 1495 is the highest among open-source models.\n\n## API Usage\n\nMiniMax-M2.7 supports two API formats:\n- **OpenAI-compatible**: `/v1/chat/completions` endpoint\n- **Anthropic-compatible**: `/v1/messages` endpoint (with extended thinking support)\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.7\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M2.7\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\", \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Anthropic-Compatible Endpoint (with Extended Thinking)\n\nThe `/v1/messages` endpoint provides access to the model's reasoning process through the \"thinking\" content block. You can also use the Anthropic SDK directly by configuring the base URL.\n\n#### Using Anthropic SDK (Recommended)\n\n```\n# Configure environment variables\nexport ANTHROPIC_BASE_URL=https://api.minimax.io/anthropic\nexport ANTHROPIC_API_KEY=${YOUR_API_KEY}\n```\n\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nmessage = client.messages.create(\n    model=\"MiniMax-M2.7\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"text\",\n                    \"text\": \"Hi, how are you?\"\n                }\n            ]\n        }\n    ]\n)\n\nfor block in message.content:\n    if block.type == \"thinking\":\n        print(f\"Thinking:\\n{block.thinking}\\n\")\n    elif block.type == \"text\":\n        print(f\"Text:\\n{block.text}\\n\")\n```\n\n#### Shell (Direct API)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.7\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Solve this step by step: What is 23 * 47?\"}\n    ]\n  }'\n```\n\n#### Shell (with Streaming)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.7\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Tell me a three sentence bedtime story about a unicorn.\"}\n    ],\n    \"stream\": true\n  }'\n```\n\n#### Python (with Streaming)\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nstream = client.messages.create(\n    model=\"MiniMax-M2.7\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Hi, how are you?\"}]}\n    ],\n    stream=True,\n)\n\nfor chunk in stream:\n    if chunk.type == \"content_block_delta\":\n        if hasattr(chunk, \"delta\") and chunk.delta:\n            if chunk.delta.type == \"thinking_delta\":\n                print(chunk.delta.thinking, end=\"\", flush=True)\n            elif chunk.delta.type == \"text_delta\":\n                print(chunk.delta.text, end=\"\", flush=True)\n```\n"},"model_price":{"modelName":"MiniMaxAI/MiniMax-M2.7","cacheHit":false,"pricePer1mPromptToken":210000,"pricePer1mCompletionToken":840000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":3000,"effectiveAt":"2026-06-02T07:44:52.040682Z","endedAt":"","id":"08f3298d-4604-4ab0-b4b9-425357c73f4f","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"claude","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":210000,"outputPrice":840000,"cacheReadPrice":42000,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":60000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"a0ef7494-9dca-4c58-a764-f0dd8798884e","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-16T00:15:02.649095869Z","update_at":"2026-10-17T09:28:58.199346813Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3.1 Flash-Lite","artifact_description":"Gemini 3.1 Flash-Lite","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemini-3.1-flash-lite-preview","artifact_template_id":"a0ef7494-9dca-4c58-a764-f0dd8798884e","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Google"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.1-flash-lite-preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.1 Flash-Lite Preview\n\nGoogle Gemini 3.1 Flash-Lite Preview is a lightweight, cost-efficient model in the Gemini 3 series from Google, optimized for high-throughput and low-latency tasks. It supports text, images, video, and audio inputs, making it suitable for a wide range of AI applications where speed and efficiency are priorities. It offers a 1M token context window for input and 8K token output capacity.\n\n## API Usage\n\nYou can interact with the Gemini 3.1 Flash-Lite Preview model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.1 Flash-Lite Preview.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.1-flash-lite-preview\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n#### example for function call\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.1-flash-lite-preview\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n#### example for vision (image input)\n```\n### example for vision (image input)\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.1-flash-lite-preview\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.1-flash-lite-preview\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n#### basic generateContent\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.1-flash-lite-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.1-flash-lite-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.1-flash-lite-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.1-flash-lite-preview\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n\n### Google Vertex SDK\n\n```\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.1-flash-lite-preview:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.1-flash-lite-preview\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.1 Flash-Lite reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.1-flash-lite-preview","cacheHit":false,"pricePer1mPromptToken":250000,"pricePer1mCompletionToken":1500000,"originPricePer1mCompletionToken":1500000,"originPricePer1mPromptToken":250000,"percentageDiscount":0,"effectiveAt":"2026-05-19T20:42:22.447136Z","endedAt":"","id":"29c2d62a-a150-4fad-a184-c45efcc42924","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":250000,"outputPrice":1500000,"cacheReadPrice":25000,"cacheWritePrice":83330,"originInputPrice":250000,"originOutputPrice":1500000,"originCacheReadPrice":25000,"originCacheWritePrice":83330}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"d76fc13b-a773-4496-b6eb-ae2cba69fdfe","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-09T03:30:42.703792992Z","update_at":"2026-10-17T09:28:58.205547113Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.4-pro","artifact_description":"GPT-5.4-pro","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.4-pro","artifact_template_id":"d76fc13b-a773-4496-b6eb-ae2cba69fdfe","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":409600,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.4-pro","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI GPT-5.4-pro\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n        - [Web search](#web-search)\n\nGPT-5.4 Pro is OpenAI's most advanced model, building on GPT-5.4's unified architecture with enhanced reasoning capabilities for complex, high-stakes tasks. It features a 1M+ token context window (922K input, 128K output) with support for text and image inputs. Optimized for step-by-step reasoning, instruction following, and accuracy, GPT-5.4 Pro excels at agentic coding, long-context workflows, and multi-step problem solving.\n## API Usage\n\nYou can interact with the OpenAI GPT-5.4-pro model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT-5.4-pro.\n\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n#### Web search\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4-pro\",\n    \"tools\": [{ \"type\": \"web_search_preview\" }],\n    \"input\": \"What was a positive news story from today?\"\n  }'\n```"},"model_price":{"modelName":"openai/gpt-5.4-pro","cacheHit":false,"pricePer1mPromptToken":30000000,"pricePer1mCompletionToken":180000000,"originPricePer1mCompletionToken":180000000,"originPricePer1mPromptToken":30000000,"percentageDiscount":0,"effectiveAt":"2026-03-14T12:35:00Z","endedAt":"","id":"769395db-8c73-417c-946a-009edc816928","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":30000000,"outputPrice":180000000,"originInputPrice":30000000,"originOutputPrice":180000000},{"threshold":272000,"inputPrice":60000000,"outputPrice":270000000,"originInputPrice":60000000,"originOutputPrice":270000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"e56e1d06-70cb-440f-b0d1-034bc2810ff5","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-07T12:14:55.046805729Z","update_at":"2026-10-17T09:28:58.212396273Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-5.4","artifact_description":"GPT-5.4","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-5.4","artifact_template_id":"e56e1d06-70cb-440f-b0d1-034bc2810ff5","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":409600,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-5.4","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI GPT-5.4\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n        - [Web search](#web-search)\n\nGPT-5.4 is OpenAI’s latest frontier model, unifying the Codex and GPT lines into a single system. It features a 1M+ token context window (922K input, 128K output) with support for text and image inputs, enabling high-context reasoning, coding, and multimodal analysis within the same workflow.\n\nThe model delivers improved performance in coding, document understanding, tool use, and instruction following. It is designed as a strong default for both general-purpose tasks and software engineering, capable of generating production-quality code, synthesizing information across multiple sources, and executing complex multi-step workflows with fewer iterations and greater token efficiency.\n## API Usage\n\nYou can interact with the OpenAI GPT-5.4 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT-5.4.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(response.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-5.4\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-5.4\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n### Create a model response\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n#### Reasoning\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this image?\"},\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n#### File Input\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"input_text\", \"text\": \"what is in this file?\"},\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n#### Web search\n```\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-5.4\",\n    \"tools\": [{ \"type\": \"web_search_preview\" }],\n    \"input\": \"What was a positive news story from today?\"\n  }'\n```"},"model_price":{"modelName":"openai/gpt-5.4","cacheHit":false,"pricePer1mPromptToken":2500000,"pricePer1mCompletionToken":15000000,"originPricePer1mCompletionToken":15000000,"originPricePer1mPromptToken":2500000,"percentageDiscount":0,"effectiveAt":"2026-03-14T12:35:00Z","endedAt":"","id":"f21597d9-7f51-4332-b3b1-808c18fc791a","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2500000,"outputPrice":15000000,"cacheReadPrice":250000,"originInputPrice":2500000,"originOutputPrice":15000000,"originCacheReadPrice":250000},{"threshold":272000,"inputPrice":5000000,"outputPrice":22500000,"cacheReadPrice":500000,"originInputPrice":5000000,"originOutputPrice":22500000,"originCacheReadPrice":500000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"4078d9c2-e385-424c-af4b-2753b94d92ff","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-02-19T20:43:36.995515403Z","update_at":"2026-10-17T09:28:58.268175862Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini-3.1-Pro","artifact_description":"gemini 3.1 pro","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemini-3.1-pro-preview","artifact_template_id":"4078d9c2-e385-424c-af4b-2753b94d92ff","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Google","is_open_source":false},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3.1-pro-preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3.1 Pro Preview\n\nGoogle Gemini 3.1 Pro Preview is the latest iteration in the Gemini 3 series from Google, featuring significantly improved reasoning capabilities for complex problem-solving tasks. Like Gemini 3 Pro, it supports text, images, video, audio, and PDF inputs, making it highly versatile for various AI applications. It offers a 1M token context window for input and 64K token output capacity.\n\n## API Usage\n\nYou can interact with the Gemini 3.1 Pro Preview model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3.1 Pro Preview.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.1-pro-preview\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n#### example for function call\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3.1-pro-preview\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n#### example for vision (image input)\n```\n### example for vision (image input)\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.1-pro-preview\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3.1-pro-preview\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n#### basic generateContent\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.1-pro-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.1-pro-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3.1-pro-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won the euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3.1-pro-preview\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n\n### Google Vertex SDK\n\n```\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3.1-pro-preview:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3.1-pro-preview\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3.1 Pro reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-3.1-pro-preview","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":12000000,"originPricePer1mCompletionToken":12000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-02-19T09:00:04.98Z","endedAt":"","id":"019931e9-5e88-4bb4-b7e8-8c2148148940","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"0113381a-3b4b-4b39-8722-ed4b4adca6f4","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.8:ray_fix_otel_v2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-21T23:11:55.356928533Z","update_at":"2026-10-27T06:45:31.237083661Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-4.7-Flash","artifact_description":"ZAI: GLM-4.7-Flash","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/zai-org/GLM-4.7-Flash","artifact_template_id":"0113381a-3b4b-4b39-8722-ed4b4adca6f4","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"bf16","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-4.7-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"12","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"12","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-4.7-Flash","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"zai-org/GLM-4.7-Flash","is_hidden":true},{"key":"RATE_LIMIT","value":"10000000","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-4.7-Flash --tp-size 2 --tool-call-parser glm47 --reasoning-parser glm45  --mem-fraction-static 0.8 --served-model-name zai-org/GLM-4.7-Flash --port 8000 --trust-remote-code","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-4.7-Flash\n\nGLM-4.7-Flash is a cutting-edge 30B-A3B MoE (Mixture of Experts) large language model developed by Z.ai. As the strongest model in the 30B class, it offers a new option for lightweight deployment that balances performance and efficiency. The model excels at reasoning, code generation, and multilingual capabilities while maintaining excellent cost-effectiveness. GLM-4.7-Flash is optimized for general-purpose AI applications such as chat, content creation, programming assistance, and knowledge retrieval.\n\n## API Usage\n\nYou can interact with the GLM-4.7-Flash model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-4.7-Flash model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-4.7-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-4.7-Flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\n                        \"symbol\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```\n\n## Resources\n\n- [HuggingFace Model Page](https://huggingface.co/zai-org/GLM-4.7-Flash)\n\n"},"model_price":{"modelName":"zai-org/GLM-4.7-Flash","cacheHit":false,"pricePer1mPromptToken":70000,"pricePer1mCompletionToken":400000,"originPricePer1mCompletionToken":400000,"originPricePer1mPromptToken":70000,"percentageDiscount":0,"effectiveAt":"2026-01-20T08:38:04.98Z","endedAt":"","id":"2196d6c5-b754-4bd6-af10-580ab3e324ce","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"1ed90614-96f6-406a-be34-f818210d7fe6","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.7:ray_fix_otel_v0","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-04T07:36:08.464190691Z","update_at":"2026-10-27T06:45:31.271137622Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMax-M2.1","artifact_description":"MiniMaxAI/MiniMax-M2.1","artifact_tags":["LLM","Text-to-Text","MiniMax"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M2.1","artifact_template_id":"1ed90614-96f6-406a-be34-f818210d7fe6","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":196608,"quantization":"fp8","info_config_map":[],"provider":""},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M2.1","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"MiniMaxAI/MiniMax-M2.1","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path MiniMaxAI/MiniMax-M2.1 --tp-size 8 --ep-size 8 --mem-fraction-static 0.85 --trust-remote-code --context-length 196608 --tool-call-parser minimax-m2 --reasoning-parser minimax-append-think --grammar-backend xgrammar --port 8000 --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 8}' --max-running-requests 16 --max-queued-requests 16","is_hidden":true}]},"artifact_details":{"model_description":"# MiniMax-M2.1\n\nM2.1 was built to shatter the stereotype that high-performance agents must remain behind closed doors. We have optimized the model specifically for robustness in coding, tool use, instruction following, and long-horizon planning. From automating multilingual software development to executing complex, multi-step office workflows, MiniMax-M2.1 empowers developers to build the next generation of autonomous applications—all while being fully transparent, controllable, and accessible.\n\n## API Usage\n\nYou can interact with the MiniMax-M2.1 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of MiniMax-M2.1.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.1\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 512\n  }'\n```\n####  example for function call\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"\"MiniMaxAI/MiniMax-M2.1\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.\u003cprovider\u003e.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M2.1\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\",   \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 512\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"MiniMaxAI/MiniMax-M2.1","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":1200000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:15:36.176154Z","endedAt":"","id":"e5d5f588-e560-4102-a394-a5b873ecba2e","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":300000,"outputPrice":1200000,"cacheReadPrice":30000,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":30000}]},"endpoints":[]},{"artifact_id":"c31a8152-aebe-4f87-8066-a0d2cecad09e","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.8:ray_fix_otel_v2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-12-23T06:50:44.311059302Z","update_at":"2026-10-27T06:45:31.288827717Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-4.7-FP8","artifact_description":"ZAI: GLM-4.7-FP8","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/zai-org/GLM-4.7-FP8","artifact_template_id":"c31a8152-aebe-4f87-8066-a0d2cecad09e","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-4.7-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"12","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"12","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-4.7-FP8","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"zai-org/GLM-4.7-FP8","is_hidden":true},{"key":"RATE_LIMIT","value":"10000000","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-4.7-FP8 --tp-size 4 --trust-remote-code --tool-call-parser glm47 --reasoning-parser glm45 --mem-fraction-static 0.85 --served-model-name zai-org/GLM-4.7-FP8 --port 8000 --enable-hierarchical-cache --hicache-ratio 2 --grammar-backend xgrammar --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}' --max-running-requests 32 --max-queued-requests 12","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-4.7\n\nGLM-4.7 is a cutting-edge large language model developed for high-performance natural language understanding and generation. It offers strong reasoning, code generation, and multilingual capabilities while maintaining excellent efficiency and cost-effectiveness. The model is optimized for general-purpose AI applications such as chat, content creation, programming assistance, and knowledge retrieval.\n\n\n## API Usage\n\nYou can interact with the GLM-4.7 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-4.7 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-4.7-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-4.7-FP8\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\n                        \"symbol\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```"},"model_price":{"modelName":"zai-org/GLM-4.7-FP8","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":2200000,"originPricePer1mCompletionToken":2200000,"originPricePer1mPromptToken":600000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:13:29.58179Z","endedAt":"","id":"62a22cdc-8210-42f8-a4b7-586fccee1f17","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":600000,"outputPrice":2200000,"cacheReadPrice":110000,"originInputPrice":600000,"originOutputPrice":2200000,"originCacheReadPrice":110000}]},"endpoints":[]},{"artifact_id":"069c106c-3275-49d0-9369-3827414a6cc1","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v5_deepseek_v32","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-12-01T12:48:03.737515519Z","update_at":"2026-10-27T06:45:31.322152549Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V3.2-Speciale","artifact_description":"deepseek-ai/DeepSeek-V3.2-Speciale","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V3.2-Speciale","artifact_template_id":"069c106c-3275-49d0-9369-3827414a6cc1","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":163840,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V3.2-Speciale","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"24","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"32","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"24","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V3.2-Speciale","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.2-Speciale --tp-size 8 --mem-fraction-static 0.9 --trust-remote-code --context-length 163840 --reasoning-parser deepseek-v3 --grammar-backend xgrammar --port 8000 --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}' --max-running-requests 32 --max-queued-requests 32","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V3.2-Speciale\n\nWe introduce **DeepSeek-V3.2**, a model that harmonizes high computational efficiency with superior reasoning and agent performance. Our approach is built upon three key technical breakthroughs:\n\n1. DeepSeek Sparse Attention (DSA): We introduce DSA, an efficient attention mechanism that substantially reduces computational complexity while preserving model performance, specifically optimized for long-context scenarios.\n\n2. Scalable Reinforcement Learning Framework: By implementing a robust RL protocol and scaling post-training compute, DeepSeek-V3.2 performs comparably to GPT-5. Notably, our high-compute variant, DeepSeek-V3.2-Speciale, surpasses GPT-5 and exhibits reasoning proficiency on par with Gemini-3.0-Pro.\n- Achievement: 🥇 Gold-medal performance in the 2025 International Mathematical Olympiad (IMO) and International Olympiad in Informatics (IOI).\n3. Large-Scale Agentic Task Synthesis Pipeline: To integrate reasoning into tool-use scenarios, we developed a novel synthesis pipeline that systematically generates training data at scale. This facilitates scalable agentic post-training, improving compliance and generalization in complex interactive environments.\n\n## API Usage\n\nYou can interact with **DeepSeek-V3.2** through the standard DeepSeek Chat Completions API endpoint.  \nBelow are example usages for shell (cURL) and Python.\n\n\n## API Examples\n\n### Example 1: Basic Chat Completion\n\nGenerate a conversational response from DeepSeek-V3.2-Speciale.\n\n#### Shell\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.2-Speciale\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500,\n    \"reasoning\": {\n      \"enabled\": true\n    }\n  }'\n```\n\n#### Python SDK usage\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.2-Speciale\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V3.2-Speciale","cacheHit":false,"pricePer1mPromptToken":280000,"pricePer1mCompletionToken":400000,"originPricePer1mCompletionToken":400000,"originPricePer1mPromptToken":280000,"percentageDiscount":0,"effectiveAt":"2025-12-02T04:38:04.98Z","endedAt":"","id":"b3c363a8-fb40-4b47-b860-11e5469fcb23","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"b17c5a1b-7906-4c82-a041-a1789b55ff33","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.8:ray_fix_otel_v5_kimi","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-10T12:02:17.818713625Z","update_at":"2026-10-27T06:45:31.43103991Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Kimi-K2-Thinking","artifact_description":"Moonshotai:Kimi-K2-Thinking\n","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/moonshotai/Kimi-K2-Thinking","artifact_template_id":"b17c5a1b-7906-4c82-a041-a1789b55ff33","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/moonshotai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"int4","info_config_map":[],"provider":"Moonshot"},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/Kimi-K2-Thinking","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"moonshotai/Kimi-K2-Thinking","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path moonshotai/Kimi-K2-Thinking --mem-fraction-static 0.90 --tp-size 8 --trust-remote-code  --tool-call-parser kimi_k2 --reasoning-parser kimi_k2 --port 8000  --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 48}' --max-running-requests 16 --max-queued-requests 16","is_hidden":true}]},"artifact_details":{"model_description":"# Moonshotai Kimi K2 Thinking\n\nKimi K2 Thinking is Moonshot AI’s most advanced open reasoning model to date, extending the K2 series into agentic, long-horizon reasoning. Built on the trillion-parameter Mixture-of-Experts (MoE) architecture introduced in Kimi K2, it activates 32 billion parameters per forward pass and supports 256 k-token context windows. The model is optimized for persistent step-by-step thought, dynamic tool invocation, and complex reasoning workflows that span hundreds of turns. It interleaves step-by-step reasoning with tool use, enabling autonomous research, coding, and writing that can persist for hundreds of sequential actions without drift.\n\nIt sets new open-source benchmarks on HLE, BrowseComp, SWE-Multilingual, and LiveCodeBench, while maintaining stable multi-agent behavior through 200–300 tool calls. Built on a large-scale MoE architecture with MuonClip optimization, it combines strong reasoning depth with high inference efficiency for demanding agentic and analytical tasks.\n\n## API Usage\n\nYou can interact with the Moonshotai Kimi K2 Instruct 0905 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Moonshotai Kimi K2 Instruct 0905.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2-Thinking\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"moonshotai/Kimi-K2-Thinking\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"moonshotai/Kimi-K2-Thinking","cacheHit":false,"pricePer1mPromptToken":800000,"pricePer1mCompletionToken":1200000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":800000,"percentageDiscount":0,"effectiveAt":"2025-11-08T04:38:04.98Z","endedAt":"","id":"54ddc322-997f-43e6-a7cd-2c187b9e8e6b","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"bbfb2cb6-7d62-43d2-a70c-f01dbdc33bde","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-05T07:37:45.615548528Z","update_at":"2026-10-27T06:45:31.451143515Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMax-M2","artifact_description":"MiniMaxAI/MiniMax-M2","artifact_tags":["LLM","Text-to-Text","MiniMax"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M2","artifact_template_id":"bbfb2cb6-7d62-43d2-a70c-f01dbdc33bde","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":196608,"quantization":"fp8","info_config_map":[],"provider":""},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M2","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"MiniMaxAI/MiniMax-M2","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path MiniMaxAI/MiniMax-M2 --tp-size 4 --mem-fraction-static 0.9 --trust-remote-code --context-length 196608 --tool-call-parser minimax-m2 --reasoning-parser minimax-append-think --port 8000 --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 8}' --max-running-requests 16 --max-queued-requests 16","is_hidden":true}]},"artifact_details":{"model_description":"# MiniMax-M2\n\nMiniMax-M2 is a compact yet high-performance large language model (LLM) designed for coding, agentic workflows, and developer-centric use cases. It brings a “mini” activation footprint while preserving “max” intelligence for real-world tasks.\n\n## API Usage\n\nYou can interact with the MiniMax-M2 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of MiniMax-M2.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 512\n  }'\n```\n####  example for function call\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"\"MiniMaxAI/MiniMax-M2\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.\u003cprovider\u003e.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M2\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\",   \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 512\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"MiniMaxAI/MiniMax-M2","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":1200000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2025-11-10T07:59:59.98Z","endedAt":"","id":"baeccbd0-bef8-42e9-8998-8b89493d899f","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"9dfb44f6-d52d-4793-a504-98b6ab95531c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_deepseek_v3.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-03T03:06:14.824040738Z","update_at":"2026-10-27T06:45:31.472282019Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek V3.1","artifact_description":"DeepSeek-V3.1","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V3.1","artifact_template_id":"9dfb44f6-d52d-4793-a504-98b6ab95531c","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":163840,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V3.1","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"32","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V3.1","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.1 --tp-size 8 --mem-fraction-static 0.85 --trust-remote-code --context-length 163840 --tool-call-parser deepseekv31 --reasoning-parser deepseek-v3 --grammar-backend xgrammar --speculative-num-steps 2 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2 --speculative-algorithm EAGLE --served-model-name deepseek-ai/DeepSeek-V3.1 --port 8000 --chat-template ./chat_template/tool_chat_template_deepseekv31.jinja --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 48}'  --max-running-requests 48 --max-queued-requests 48","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V3.1\n\nDeepSeek-V3.1 is a powerful large language model maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-V3.1 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V3.1.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.1\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V3.1\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.1\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V3.1","cacheHit":false,"pricePer1mPromptToken":270000,"pricePer1mCompletionToken":1000000,"originPricePer1mCompletionToken":1000000,"originPricePer1mPromptToken":270000,"percentageDiscount":0,"effectiveAt":"2025-08-21T08:38:04.98Z","endedAt":"","id":"d1855d17-bde1-4a84-873f-86785a423938","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"11e2d1e4-2fb7-49a7-b625-fe9e41acbb72","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-09-30T18:14:15.918642091Z","update_at":"2026-10-27T06:45:31.568268014Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GLM-4.6","artifact_description":"ZAI: GLM-4.6","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/zai-org/GLM-4.6","artifact_template_id":"11e2d1e4-2fb7-49a7-b625-fe9e41acbb72","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":204800,"quantization":"bf16","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-4.6","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-4.6","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-4.6 --tp-size 8 --mem-fraction-static 0.8 --trust-remote-code --tool-call-parser glm45 --reasoning-parser glm45 --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-4.6\n\nGLM-4.6 is a cutting-edge large language model developed for high-performance natural language understanding and generation. It offers strong reasoning, code generation, and multilingual capabilities while maintaining excellent efficiency and cost-effectiveness. The model is optimized for general-purpose AI applications such as chat, content creation, programming assistance, and knowledge retrieval.\n\n\n## API Usage\n\nYou can interact with the GLM-4.6 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-4.6 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-4.6\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-4.6\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\n                        \"symbol\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```"},"model_price":{"modelName":"zai-org/GLM-4.6","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":2000000,"originPricePer1mCompletionToken":2000000,"originPricePer1mPromptToken":600000,"percentageDiscount":0,"effectiveAt":"2025-10-01T17:33:59.813Z","endedAt":"","id":"9cd00758-74af-45fb-abb8-899a20aca489","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"3ae3ce59-178d-4f38-893b-d94315804579","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15_ds3.2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-09-29T19:10:33.115153089Z","update_at":"2026-10-27T06:45:31.575258252Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V3.2-Exp","artifact_description":"deepseek-ai/DeepSeek-V3.2-Exp","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V3.2-Exp","artifact_template_id":"3ae3ce59-178d-4f38-893b-d94315804579","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":163840,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":true,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V3.2-Exp","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V3.2-Exp","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.2-Exp --tensor-parallel-size 8 --mem-fraction-static 0.85 --trust-remote-code --context-length 163840 --tool-call-parser deepseekv31 --reasoning-parser deepseek-v3 --port 8000 --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}'","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V3.2-Exp\n\n**DeepSeek-V3.2-Exp** is the latest experimental release in the DeepSeek model series, designed for enhanced reasoning, contextual understanding, and long-context dialogue handling.  \nCompared to previous versions, **V3.2-Exp** introduces improved efficiency in inference and optimized attention mechanisms, making it ideal for both research and production-grade AI applications.\n\nThis model continues to provide **high-quality, consistent responses** across diverse tasks — from natural language understanding and reasoning to code generation and function calling — while maintaining a strong balance between performance and computational efficiency.\n\n\n## Model Information\n\n- **Model Path:** `deepseek-ai/DeepSeek-V3.2-Exp`\n- **Release Type:** Experimental (Enhanced Contextual and Reasoning Capabilities)\n\n### Key Improvements Over V3.1\n- Extended context length and improved long-range coherence.\n- More stable and accurate tool/function calling.\n- Enhanced reasoning and factual accuracy in complex tasks.\n- Improved response efficiency and output determinism under low temperatures.\n\n\n## API Usage\n\nYou can interact with **DeepSeek-V3.2-Exp** through the standard DeepSeek Chat Completions API endpoint.  \nBelow are example usages for shell (cURL) and Python.\n\n\n## API Examples\n\n### Example 1: Basic Chat Completion\n\nGenerate a conversational response from DeepSeek-V3.2-Exp.\n\n#### Shell\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.2-Exp\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n##### Function Call\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V3.2-Exp\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get the weather for a given city. The user must provide a city name.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city name, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\"city\"]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"How's the weather in Qingdao today?\"\n        }\n    ]\n}'\n```\n\n#### Python SDK usage\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.2-Exp\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V3.2-Exp","cacheHit":false,"pricePer1mPromptToken":270000,"pricePer1mCompletionToken":410000,"originPricePer1mCompletionToken":410000,"originPricePer1mPromptToken":270000,"percentageDiscount":0,"effectiveAt":"2025-09-29T08:38:04.98Z","endedAt":"","id":"7806fb30-7792-4621-8ef5-ad365766f55b","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"11e7a66b-3eac-4e3b-9c32-39bd263868d8","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_0.11.2:otel_v5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-25T13:19:29.602627285Z","update_at":"2026-10-27T06:45:31.35628874Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"allenai: olmOCR-2-7B-1025-FP8","artifact_description":"This is a release of the olmOCR model that's fine tuned from Qwen2.5-VL-7B-Instruct using the olmOCR-mix-1025 dataset. It has been additionally fine tuned using GRPO RL training to boost its performance at math equations, tables, and other tricky OCR cases.","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/allenai/olmOCR-2-7B-1025-FP8","artifact_template_id":"11e7a66b-3eac-4e3b-9c32-39bd263868d8","artifact_icon_link":"https://cdn-uploads.huggingface.co/production/uploads/6734d6722769638944a5aa2e/DPsr3ZvRF9v-gdMa4EaHW.png","preview_image_url":"","is_public":true,"model_info":{"context_length":128000,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/allenai/olmOCR-2-7B-1025-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"30","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"40","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"allenai/olmOCR-2-7B-1025-FP8","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"allenai/olmOCR-2-7B-1025-FP8","is_hidden":true},{"key":"RATE_LIMIT","value":"10000000","is_hidden":true},{"key":"SERVE_COMMAND","value":"vllm serve allenai/olmOCR-2-7B-1025-FP8 --max-model-len 128000 --port 8000 ","is_hidden":true}]},"artifact_details":{"model_description":"# olmOCR-2-7B-1025-FP8\n\nQuantized to FP8 Version of olmOCR-2-7B-1025, using llmcompressor.\n\nThis is a release of the olmOCR model that's fine tuned from Qwen2.5-VL-7B-Instruct using the olmOCR-mix-1025 dataset. It has been additionally fine tuned using GRPO RL training to boost its performance at math equations, tables, and other tricky OCR cases.\n\n\n## API Usage\n\nYou can interact with the olmOCR-2-7B-1025 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of olmOCR-2-7B-1025.\n\n### Shell\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer *************\" \\\n    -d '{\n  \"model\": \"allenai/olmOCR-2-7B-1025-FP8\",\n  \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://olmocr.allenai.org/boxplotswhite-4t0gsafj.png\"\n            }\n          }\n        ]\n      }\n    ],\n  \"temperature\": 0,\n  \"max_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"allenai/olmOCR-2-7B-1025-FP8\",\n    \"messages\": [\n        {\n          \"role\": \"user\",\n          \"content\": [\n            {\n              \"type\": \"text\",\n              \"text\": \"What is in this image?\"\n            },\n            {\n              \"type\": \"image_url\",\n              \"image_url\": {\n                \"url\": \"https://olmocr.allenai.org/boxplotswhite-4t0gsafj.png\"\n              }\n            }\n          ]\n        }\n      ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Openai-python\n```\nimport os\nfrom openai import OpenAI\n\nclient = OpenAI(\n  base_url=\"https://api.gmi-serving.com/v1\",\n  api_key=\"\u003cGMI_API_KEY\u003e\",\n)\n\ncompletion = client.chat.completions.create(\n  model=\"allenai/olmOCR-2-7B-1025-FP8\",\n  messages=[\n              {\n                \"role\": \"user\",\n                \"content\": [\n                  {\n                    \"type\": \"text\",\n                    \"text\": \"What is in this image? give me more details\"\n                  },\n                  {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                      \"url\": \"https://olmocr.allenai.org/boxplotswhite-4t0gsafj.png\"\n                    }\n                  }\n                ]\n              }\n            ]\n)\n\nprint(completion.choices[0].message.content)\n```"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"19a10465-f028-46cd-bd6f-91921c886699","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-20T03:20:48.069099416Z","update_at":"2026-10-17T09:28:58.422632306Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"deepseek-ai/DeepSeek-R1-0528","artifact_description":"deepseek-ai/DeepSeek-R1-0528","artifact_tags":["LLM"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-0528","artifact_template_id":"19a10465-f028-46cd-bd6f-91921c886699","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-0528","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# DeepSeek-V3.2\n\nDeepSeek-V3.2 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek R1 0528 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V3.2.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.2\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.2\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1-0528","cacheHit":false,"pricePer1mPromptToken":570000,"pricePer1mCompletionToken":2290000,"originPricePer1mCompletionToken":2290000,"originPricePer1mPromptToken":570000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:10:50.554509Z","endedAt":"","id":"370763e4-b710-4194-b0fd-067b783cf3c8","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":570000,"outputPrice":2290000,"cacheReadPrice":230000,"originInputPrice":570000,"originOutputPrice":2290000,"originCacheReadPrice":230000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"8d65e8ed-986d-45c2-b417-95ba8dce6605","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-21T09:36:42.162410688Z","update_at":"2026-10-17T09:28:58.394625937Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"deepseek-ai/DeepSeek-V3.2","artifact_description":"deepseek-ai/DeepSeek-V3.2","artifact_tags":["LLM"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V3.2","artifact_template_id":"8d65e8ed-986d-45c2-b417-95ba8dce6605","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":163840,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V3.2","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# DeepSeek V3.2\n\nDeepSeek V3.2 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek V3.2 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek V3.2.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.2\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.2\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V3.2","cacheHit":false,"pricePer1mPromptToken":208800,"pricePer1mCompletionToken":309600,"originPricePer1mCompletionToken":430000,"originPricePer1mPromptToken":290000,"percentageDiscount":2800,"effectiveAt":"2026-04-10T06:12:31.734313Z","endedAt":"","id":"cc1ce9ae-b656-4955-bd73-b26f50839f54","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":208800,"outputPrice":309600,"cacheReadPrice":21600,"originInputPrice":290000,"originOutputPrice":430000,"originCacheReadPrice":30000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"57bc0854-4e0b-435b-a316-28519301723c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.6","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-04-30T09:06:29.231439952Z","update_at":"2026-10-27T06:45:31.831283375Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek: DeepSeek Prover V2 671B","artifact_description":"deepseek-ai/DeepSeek-Prover-V2-671B","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-Prover-V2-671B","artifact_template_id":"57bc0854-4e0b-435b-a316-28519301723c","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":null,"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-Prover-V2-671B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-Prover-V2-671B --tp-size 8  --mem-fraction-static 0.8 --trust-remote-code --context-length 131072 --tool-call-parser deepseekv3 --attention-backend fa3 --port 8000 --speculative-algorithm EAGLE --speculative-num-steps 2 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2 --cpu-offload-gb 1000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"SGL_ENABLE_JIT_DEEPGEMM","value":"1","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"20","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"20","is_hidden":false},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false}]},"artifact_details":{"model_description":"# DeepSeek Prover V2 671B\n\nDeepSeek Prover V2 671B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek Prover V2 671B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek Prover V2 671B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-Prover-V2-671B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-Prover-V2-671B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-Prover-V2-671B","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":2180000,"originPricePer1mCompletionToken":2180000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2026-05-09T04:09:00Z","endedAt":"","id":"a147ddda-5a4a-4cca-9316-02241608e741","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"56f9685a-99a9-4e9e-9e03-1b7de29fc973","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.6.post2:v1.5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-20T20:47:55.77809-08:00","update_at":"2026-11-03T19:51:45.417200516Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1","artifact_description":"DeepSeek-R1","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1","artifact_template_id":"56f9685a-99a9-4e9e-9e03-1b7de29fc973","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"SGL_ENABLE_JIT_DEEPGEMM","value":"1","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1 --tp-size 8  --mem-fraction-static 0.9 --trust-remote-code --context-length 131072 --tool-call-parser deepseekv3 --reasoning-parser deepseek-r1 --attention-backend fa3 --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek R1\n\nDeepSeek R1 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek R1 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek R1.\n \n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":2180000,"originPricePer1mCompletionToken":2180000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:51.689126Z","endedAt":"","id":"aab8f89b-2663-4b6b-898e-df15ddbf7276","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"c271c9b8-621b-4f5d-ab1c-7725506bb2c5","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.6.post2:v1.5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-28T20:21:50.109609465Z","update_at":"2026-10-27T06:45:31.881508672Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1 Distill Llama 70B","artifact_description":"DeepSeek-R1-Distill-Llama-70B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Llama-70B","artifact_template_id":"c271c9b8-621b-4f5d-ab1c-7725506bb2c5","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Llama-70B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Llama-70B --trust-remote-code --context-length 131072 --mem-fraction-static 0.9 --tensor-parallel-size 2 --tool-call-parser llama3 --reasoning-parser deepseek-r1 --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-R1-Distill-Llama-70B\n\nDeepSeek-R1-Distill-Llama-70B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-R1-Distill-Llama-70B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-R1-Distill-Llama-70B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Llama-70B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Llama-70B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1-Distill-Llama-70B","cacheHit":false,"pricePer1mPromptToken":250000,"pricePer1mCompletionToken":750000,"originPricePer1mCompletionToken":750000,"originPricePer1mPromptToken":250000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:53.832185Z","endedAt":"","id":"087cb674-fbd9-4cb7-9d38-10390b7b7cca","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"97926200-2bd6-45d4-9c3f-a2c6b4c0e8bf","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.6.post2:v1.5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-20T17:50:14.84202-08:00","update_at":"2026-10-27T06:45:31.967475859Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1 Distill Llama 8B","artifact_description":"DeepSeek R1 Distill Llama 8B","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Llama-8B","artifact_template_id":"97926200-2bd6-45d4-9c3f-a2c6b4c0e8bf","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Llama-8B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Llama-8B --trust-remote-code --context-length 131072 --mem-fraction-static 0.8 --tensor-parallel-size 1 --tool-call-parser llama3 --reasoning-parser deepseek-r1 --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek R1 Distill Llama 8B\n\nDeepSeek R1 Distill Llama 8B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek R1 Distill Llama 8B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek R1 Distill Llama 8B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Llama-8B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Llama-8B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1-Distill-Llama-8B","cacheHit":false,"pricePer1mPromptToken":140000,"pricePer1mCompletionToken":390000,"originPricePer1mCompletionToken":390000,"originPricePer1mPromptToken":140000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:53.525489Z","endedAt":"","id":"88757678-e823-41af-9b6f-2d6e2f38df2b","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"daa868d8-5e41-4f11-b14b-b130ffeac43e","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-28T20:21:03.839875734Z","update_at":"2026-10-27T06:45:31.88831381Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1 Distill Qwen 14B","artifact_description":"DeepSeek-R1-Distill-Qwen-14B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B","artifact_template_id":"daa868d8-5e41-4f11-b14b-b130ffeac43e","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-14B --mem-fraction-static 0.8 --trust-remote-code --context-length 131072 --tool-call-parser deepseekv3 --reasoning-parser deepseek-r1 --enable-metrics --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-R1-Distill-Qwen-14B\n\nDeepSeek-R1-Distill-Qwen-14B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-R1-Distill-Qwen-14B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-R1-Distill-Qwen-14B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-14B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-14B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1-Distill-Qwen-14B","cacheHit":false,"pricePer1mPromptToken":200000,"pricePer1mCompletionToken":200000,"originPricePer1mCompletionToken":200000,"originPricePer1mPromptToken":200000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:52.834509Z","endedAt":"","id":"665f1ad2-26c2-4258-8c35-49d96c9692ec","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"4f4ad4ff-7763-4970-9736-d9d1018fd7b5","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-28T20:20:21.625406574Z","update_at":"2026-10-27T06:45:31.902213636Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1 Distill Qwen 1.5B","artifact_description":"DeepSeek-R1-Distill-Qwen-1.5B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B","artifact_template_id":"4f4ad4ff-7763-4970-9736-d9d1018fd7b5","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B --mem-fraction-static 0.8 --trust-remote-code --context-length 131072 --tool-call-parser qwen25 --reasoning-parser deepseek-r1 --enable-metrics --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-R1-Distill-Qwen-1.5B\n\nDeepSeek-R1-Distill-Qwen-1.5B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-R1-Distill-Qwen-1.5B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-R1-Distill-Qwen-1.5B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"215bfbc7-5ccb-40fa-af53-1c75e1554b1d","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-20T20:46:42.267881-08:00","update_at":"2026-10-27T06:45:31.960785141Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1 Distill Qwen 32B","artifact_description":"DeepSeek-R1-32B","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B","artifact_template_id":"215bfbc7-5ccb-40fa-af53-1c75e1554b1d","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"5","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-32B --mem-fraction-static 0.8 --trust-remote-code --context-length 131072 --tool-call-parser deepseekv3 --reasoning-parser deepseek-r1 --enable-metrics --allow-auto-truncate --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-R1-Distill-Qwen-32B\nDeepSeek-R1-Distill-Qwen-32B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-R1-Distill-Qwen-32B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-R1-Distill-Qwen-32B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-32B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-32B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1-Distill-Qwen-32B","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":900000,"originPricePer1mCompletionToken":900000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:53.222719Z","endedAt":"","id":"15957cc7-5bf5-4f17-bcea-dabe15f8a917","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"64e7dbd9-739a-4e6e-a9e1-c4abf4a79d83","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-02-28T20:20:48.190355093Z","update_at":"2026-10-27T06:45:31.895250828Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek R1 Distill Qwen 7B","artifact_description":"DeepSeek-R1-Distill-Qwen-7B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B","artifact_template_id":"64e7dbd9-739a-4e6e-a9e1-c4abf4a79d83","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-R1-Distill-Qwen-7B --mem-fraction-static 0.8 --trust-remote-code --context-length 131072 --tool-call-parser deepseekv3 --reasoning-parser deepseek-r1 --enable-metrics --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-R1-Distill-Qwen-7B\n\nDeepSeek-R1-Distill-Qwen-7B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-R1-Distill-Qwen-7B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-R1-Distill-Qwen-7B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-7B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-R1-Distill-Qwen-7B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-R1-Distill-Qwen-7B","cacheHit":false,"pricePer1mPromptToken":100000,"pricePer1mCompletionToken":200000,"originPricePer1mCompletionToken":200000,"originPricePer1mPromptToken":100000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:52.622576Z","endedAt":"","id":"ef4596d3-f292-4f57-b96c-e15e92cca6a2","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"54540344-cd74-40e8-ade4-c84048307645","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-29T05:13:09.64496833Z","update_at":"2026-10-17T09:28:58.344425959Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek V3 0324","artifact_description":"deepseek-ai/DeepSeek-V3-0324","artifact_tags":["LLM"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V3-0324","artifact_template_id":"54540344-cd74-40e8-ade4-c84048307645","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":128000,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V3-0324","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# DeepSeek-V3-0324\n\nDeepSeek-V3-0324 is a powerful large language model maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek-V3-0324 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V3-0324.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3-0324\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V3-0324\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3-0324\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V3-0324","cacheHit":false,"pricePer1mPromptToken":290000,"pricePer1mCompletionToken":1140000,"originPricePer1mCompletionToken":1140000,"originPricePer1mPromptToken":290000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:11:40.199317Z","endedAt":"","id":"86d34ad4-d378-413d-b0e1-4f79bf068a87","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":290000,"outputPrice":1140000,"cacheReadPrice":110000,"originInputPrice":290000,"originOutputPrice":1140000,"originCacheReadPrice":110000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"07fcd6dc-f952-4550-bffa-9d793f93893a","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.14_2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-09-22T14:56:34.327691295Z","update_at":"2026-10-27T06:45:31.589418808Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V3.1-Terminus","artifact_description":"deepseek-ai/DeepSeek-V3.1-Terminus","artifact_tags":["LLM","DeepSeek","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V3.1-Terminus","artifact_template_id":"07fcd6dc-f952-4550-bffa-9d793f93893a","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V3.1-Terminus","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"15","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":true},{"key":"MIN_REPLICAS","value":"1","is_hidden":true},{"key":"MAX_REPLICAS","value":"1","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V3.1-Terminus","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V3.1-Terminus --tp-size 8  --mem-fraction-static 0.9 --trust-remote-code --context-length 131072 --tool-call-parser deepseekv31 --reasoning-parser deepseek-v3 --port 8000 --max-queued-requests 15","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V3.1-Terminus\n\n**DeepSeek-V3.1-Terminus** represents the culmination of the V3.1 series — a highly optimized large language model engineered for **maximum inference efficiency**, **stability**, and **precision**.  \nIt builds upon DeepSeek-V3.1’s strong foundational performance while introducing targeted refinements for advanced reasoning, deterministic output control, and reduced latency.\n\n\n\n## API Usage\n\nYou can access **DeepSeek-V3.1-Terminus** via the standard Chat Completions API endpoint.  \nIt supports text generation, reasoning tasks, and structured tool/function calling.\n\n\n## API Examples\n\n### Basic Chat Completion\n\nGenerate a response using the chat endpoint of DeepSeek-V3.1-Terminus.\n\n#### Shell\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.1-Terminus\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n##### Function Calling\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V3.1-Terminus\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get the weather of a city. The user must provide a city name.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city name, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\"city\"]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"How's the weather in Qingdao today?\"\n        }\n    ]\n}'\n```\n\n#### Python SDK Usage\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.1-Terminus\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V3.1-Terminus","cacheHit":false,"pricePer1mPromptToken":270000,"pricePer1mCompletionToken":1000000,"originPricePer1mCompletionToken":1000000,"originPricePer1mPromptToken":270000,"percentageDiscount":0,"effectiveAt":"2025-09-23T08:38:04.98Z","endedAt":"","id":"ffdbce45-f5d4-4c18-bbab-8a76b8a478ac","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"0049392b-5a6b-4e94-acc0-e18b8e293aac","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-10T18:31:22.69341552Z","update_at":"2026-10-17T09:28:57.702970269Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4.1-Flash","artifact_description":"deepseek-ai/DeepSeek-V4.1-Flash","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4.1-Flash","artifact_template_id":"0049392b-5a6b-4e94-acc0-e18b8e293aac","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4.1-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/DeepSeek-V4.1-Flash \n\nDeepSeek-V4.1-Flash is the official release of DeepSeek-V4-Flash, superseding the preview version. It is an efficiency-optimized Mixture-of-Experts model with 284B total parameters and 13B activated parameters, supporting a 1M-token context window. The model keeps the same architecture and size as the preview release and was re-post-trained with substantially enhanced agentic capabilities.\n\nDeepSeek-V4.1-Flash  includes hybrid attention (Compressed Sparse Attention and Heavily Compressed Attention) for efficient long-context processing, and supports configurable reasoning modes via the `reasoning_effort` parameter (`low`, `high`, and `max`). It is well suited for applications such as coding assistants, chat systems, and agent workflows where responsiveness, long-context efficiency, and cost efficiency are important.\n\n## API Usage\n\nYou can interact with the deepseek-ai/DeepSeek-V4-Flash-0731 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V4.1-Flash.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash \",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4.1-Flash","cacheHit":false,"pricePer1mPromptToken":180000,"pricePer1mCompletionToken":720000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":4000,"effectiveAt":"2026-09-10T18:40:22.301306Z","endedAt":"","id":"950b84fc-f226-462a-9780-d5c4361bfc7c","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":180000,"outputPrice":720000,"cacheReadPrice":3600,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":6000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"bf5fdc4e-0575-417e-8153-9f9558e8f898","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_deepseek_v4_1_flash:ray_fix_otel_v1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-15T13:37:11.389142477Z","update_at":"2026-10-27T06:45:30.977632759Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4.1-Flash-H200-dev","artifact_description":"DeepSeek-V4.1-Flash","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4.1-Flash","artifact_template_id":"bf5fdc4e-0575-417e-8153-9f9558e8f898","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp4","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4.1-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":false,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"32","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"64","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"12","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V4.1-Flash","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_v12_b300_dev_v4","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V4.1-Flash --served-model-name deepseek-ai/DeepSeek-V4.1-Flash --trust-remote-code --tp-size 8 --ep-size 8 --mem-fraction-static 0.8 --attention-backend dsv4 --moe-runner-backend flashinfer_mxfp4 --cuda-graph-max-bs-decode 64 --reasoning-parser auto --tool-call-parser auto --port 8000 --enable-cache-report --max-running-requests 64","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V4.1-Flash\n\nDeepSeek V4.1 Flash is a sparse mixture-of-experts model from DeepSeek, and the first built on the company's Causal Encoder-Decoder (CED) architecture. It activates 8B parameters on input and 16B on output from a 552B-parameter backbone, an asymmetric split that keeps per-token compute low relative to the model's total size. Image understanding is native to the architecture, with visual and text embeddings trained jointly from the start of pre-training rather than added afterward as in the earlier experimental V4 Flash Vision Exp.\n\n\n## API Usage\n\nYou can interact with the DeepSeek-V4.1-Flash model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the DeepSeek-V4.1-Flash model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4.1-Flash\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4.1-Flash","cacheHit":false,"pricePer1mPromptToken":180000,"pricePer1mCompletionToken":720000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":4000,"effectiveAt":"2026-09-10T18:40:22.301306Z","endedAt":"","id":"950b84fc-f226-462a-9780-d5c4361bfc7c","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":180000,"outputPrice":720000,"cacheReadPrice":3600,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":6000}]},"endpoints":[]},{"artifact_id":"16e9ccd2-84e0-43b4-9d9a-5cb31b3341a2","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-24T08:13:30.563402611Z","update_at":"2026-10-17T09:28:58.093881947Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Flash","artifact_description":"deepseek-ai/DeepSeek-V4-Flash","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Flash","artifact_template_id":"16e9ccd2-84e0-43b4-9d9a-5cb31b3341a2","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/DeepSeek-V4-Flash\n\nDeepSeek V4 Flash is an efficiency-optimized Mixture-of-Experts model from DeepSeek with 284B total parameters and 13B activated parameters, supporting a 1M-token context window. It is designed for fast inference and high-throughput workloads, while maintaining strong reasoning and coding performance.\n\nThe model includes hybrid attention for efficient long-context processing and supports configurable reasoning modes. It is well suited for applications such as coding assistants, chat systems, and agent workflows where responsiveness and cost efficiency are important.\n\n## API Usage\n\nYou can interact with the deepseek-ai/DeepSeek-V4-Flash model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V4-Flash.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Flash","cacheHit":false,"pricePer1mPromptToken":91000,"pricePer1mCompletionToken":182000,"originPricePer1mCompletionToken":280000,"originPricePer1mPromptToken":140000,"percentageDiscount":3500,"effectiveAt":"2026-08-29T04:33:08.265907Z","endedAt":"","id":"2bee945b-200c-4cda-a4bc-7a2c105a9b16","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":91000,"outputPrice":182000,"cacheReadPrice":18200,"originInputPrice":140000,"originOutputPrice":280000,"originCacheReadPrice":28000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"3cfa36ba-8fa2-47b0-b58f-16a37312a602","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_v12_b300_dev_v4","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-27T05:12:51.415779835Z","update_at":"2026-10-27T06:45:31.027324776Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Flash-0731-B300","artifact_description":"DeepSeek-V4-Flash-0731","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","artifact_template_id":"3cfa36ba-8fa2-47b0-b58f-16a37312a602","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp4","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"320","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"512","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"128","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V4-Flash-0731","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_b300_dev_v2","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V4-Flash-0731 --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 --tp-size 1 --trust-remote-code --reasoning-parser deepseek-v4 --tool-call-parser deepseekv4 --moe-runner-backend flashinfer_mxfp4 --speculative-algorithm DSPARK --port 8000 --enable-cache-report --mem-fraction-static 0.9 --max-running-requests 512","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V4-Flash-0731\n\nDeepSeek-V4-Flash-0731 is a high-performance language model optimized for fast, efficient text generation, reasoning, coding, multilingual understanding, and agentic workflows. It supports long-context applications and OpenAI-compatible chat completion APIs.\n\n\n## API Usage\n\nYou can interact with the DeepSeek-V4-Flash-0731 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the DeepSeek-V4-Flash-0731 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Flash-0731","cacheHit":false,"pricePer1mPromptToken":286000,"pricePer1mCompletionToken":858000,"originPricePer1mCompletionToken":1320000,"originPricePer1mPromptToken":440000,"percentageDiscount":3500,"effectiveAt":"2026-08-29T03:25:48.122941Z","endedAt":"","id":"8e2d13d2-d705-4b8b-9471-8e130a677756","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"B300","gpuPrice":4700000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":286000,"outputPrice":858000,"cacheReadPrice":9100,"originInputPrice":440000,"originOutputPrice":1320000,"originCacheReadPrice":14000}]},"endpoints":[]},{"artifact_id":"8726c012-e990-4134-872c-78f883f7d929","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.19:ray_fix_otel_v2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-11T15:38:47.043736044Z","update_at":"2026-10-27T06:45:31.01324722Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Flash-0731-B300-dev","artifact_description":"DeepSeek-V4-Flash-0731","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","artifact_template_id":"8726c012-e990-4134-872c-78f883f7d929","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp4","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"196","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"256","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"120","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V4-Flash-0731","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_v12_b300_dev_v4","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V4-Flash-0731 --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 --tp-size 1 --trust-remote-code --reasoning-parser deepseek-v4 --tool-call-parser deepseekv4 --moe-runner-backend flashinfer_mxfp4 --speculative-algorithm DSPARK --port 8000 --enable-cache-report --mem-fraction-static 0.9 --max-running-requests 256","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V4-Flash-0731\n\nDeepSeek-V4-Flash-0731 is a high-performance language model optimized for fast, efficient text generation, reasoning, coding, multilingual understanding, and agentic workflows. It supports long-context applications and OpenAI-compatible chat completion APIs.\n\n\n## API Usage\n\nYou can interact with the DeepSeek-V4-Flash-0731 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the DeepSeek-V4-Flash-0731 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Flash-0731","cacheHit":false,"pricePer1mPromptToken":286000,"pricePer1mCompletionToken":858000,"originPricePer1mCompletionToken":1320000,"originPricePer1mPromptToken":440000,"percentageDiscount":3500,"effectiveAt":"2026-08-29T03:25:48.122941Z","endedAt":"","id":"8e2d13d2-d705-4b8b-9471-8e130a677756","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"B300","gpuPrice":4700000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":286000,"outputPrice":858000,"cacheReadPrice":9100,"originInputPrice":440000,"originOutputPrice":1320000,"originCacheReadPrice":14000}]},"endpoints":[]},{"artifact_id":"530b7bdb-5c11-4323-ac36-bbac1ee84728","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_v11","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-07T09:01:46.810262386Z","update_at":"2026-10-27T06:45:31.059896747Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Flash-0731-dev","artifact_description":"DeepSeek-V4-Flash-0731","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","artifact_template_id":"530b7bdb-5c11-4323-ac36-bbac1ee84728","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp4","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Flash-0731","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"32","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V4-Flash-0731","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_glm5_h200:llm-engine_v3.3","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path deepseek-ai/DeepSeek-V4-Flash-0731 --served-model-name deepseek-ai/DeepSeek-V4-Flash-0731 --tp-size 8 --trust-remote-code --reasoning-parser deepseek-v4 --tool-call-parser deepseekv4 --moe-runner-backend flashinfer_mxfp4 --speculative-algorithm DSPARK --port 8000 --enable-cache-report --mem-fraction-static 0.85 --max-running-requests 128","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V4-Flash-0731\n\nDeepSeek-V4-Flash-0731 is a high-performance language model optimized for fast, efficient text generation, reasoning, coding, multilingual understanding, and agentic workflows. It supports long-context applications and OpenAI-compatible chat completion APIs.\n\n\n## API Usage\n\nYou can interact with the DeepSeek-V4-Flash-0731 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the DeepSeek-V4-Flash-0731 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Flash-0731\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Flash-0731","cacheHit":false,"pricePer1mPromptToken":286000,"pricePer1mCompletionToken":858000,"originPricePer1mCompletionToken":1320000,"originPricePer1mPromptToken":440000,"percentageDiscount":3500,"effectiveAt":"2026-08-29T03:25:48.122941Z","endedAt":"","id":"8e2d13d2-d705-4b8b-9471-8e130a677756","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":286000,"outputPrice":858000,"cacheReadPrice":9100,"originInputPrice":440000,"originOutputPrice":1320000,"originCacheReadPrice":14000}]},"endpoints":[]},{"artifact_id":"1c0d2152-07d5-4856-8930-824013203309","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-21T21:39:38.803779783Z","update_at":"2026-10-17T09:28:57.796208636Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Flash-Vision-Exp","artifact_description":"deepseek-ai/deepseek-v4-flash-vision-exp","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/deepseek-v4-flash-vision-exp","artifact_template_id":"1c0d2152-07d5-4856-8930-824013203309","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"fp8","info_config_map":[],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/deepseek-v4-flash-vision-exp","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/deepseek-v4-flash-vision-exp\n\nDeepSeek-V4-Flash-Vision-Exp is an experimental multimodal vision-language model from DeepSeek, built on the DeepSeek-V4-Flash architecture. It is a Mixture-of-Experts model with 284B total parameters and 13B activated parameters, supporting a 1M-token context window. For pure-text tasks such as reasoning, agent workflows, and world knowledge, it matches the official DeepSeek-V4-Flash, while adding native image understanding for use cases such as screenshot analysis, chart reading, OCR, and multimodal agents.\n\nThe model accepts JPEG, PNG, GIF, and WebP images alongside text. Images can be provided via base64 data URLs, external URLs, or the Files API using a `file_id`. Each image is billed at up to 384 tokens at standard DeepSeek-V4-Flash text pricing. This is an experimental release; the model identifier and API behavior may change before general availability.\n\n## API Usage\n\nYou can interact with the deepseek-ai/deepseek-v4-flash-vision-exp model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V4-Flash-Vision-Exp.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/deepseek-v4-flash-vision-exp\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for image input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/deepseek-v4-flash-vision-exp\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this image?\"},\n          {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/image.jpg\"}}\n        ]\n      }\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/deepseek-v4-flash-vision-exp\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/deepseek-v4-flash-vision-exp\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### example for image input\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/deepseek-v4-flash-vision-exp\",\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\"url\": \"https://example.com/image.jpg\"}\n                }\n            ]\n        }\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"deepseek-ai/deepseek-v4-flash-vision-exp","cacheHit":false,"pricePer1mPromptToken":440000,"pricePer1mCompletionToken":1320000,"originPricePer1mCompletionToken":1320000,"originPricePer1mPromptToken":440000,"percentageDiscount":0,"effectiveAt":"2026-08-29T03:26:48.754833Z","endedAt":"","id":"a2c11566-6451-4f9d-b1b3-3bba301c4430","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":440000,"outputPrice":1320000,"cacheReadPrice":14000,"originInputPrice":440000,"originOutputPrice":1320000,"originCacheReadPrice":14000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"6851053d-0320-4b4e-895b-52ab6afd8532","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_deepseek_v4_pro_b200:llm-engine_v1.0","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-27T21:35:59.080521068Z","update_at":"2026-10-27T06:45:31.08778571Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Pro","artifact_description":"DeepSeek-V4-Pro","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Pro","artifact_template_id":"6851053d-0320-4b4e-895b-52ab6afd8532","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/deepseek.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":65536,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document input","category":"general"}],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Pro","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"4","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"deepseek-ai/DeepSeek-V4-Pro","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_glm5_h200:llm-engine_v3.3","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SGLANG_ENABLE_SPEC_V2","value":"1","is_hidden":true},{"key":"SGLANG_NSA_FORCE_MLA","value":"1","is_hidden":true},{"key":"SGLANG_ENABLE_JIT_DEEPGEMM","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server \\\n  --model-path /mnt/model_garden \\\n  --served-model-name deepseek-ai/DeepSeek-V4-Pro \\\n  --tp-size 8 \\\n  --trust-remote-code \\\n  --reasoning-parser deepseek-v4 \\\n  --tool-call-parser deepseekv4 \\\n  --context-length 65536 \\\n  --chunked-prefill-size 4096 \\\n  --mem-fraction-static 0.82 \\\n  --max-running-requests 64 \\\n  --max-queued-requests 128 \\\n  --watchdog-timeout 3600 \\\n  --moe-runner-backend flashinfer_mxfp4 \\\n  --fp8-gemm-backend cutlass \\\n  --enable-metrics \\\n  --model-loader-extra-config '{\"enable_multithread_load\":true,\"num_threads\":16}'","is_hidden":true}]},"artifact_details":{"model_description":"# DeepSeek-V4-Pro\n\nDeepSeek-V4-Pro is a high-performance language model optimized for advanced reasoning, coding, multilingual understanding, text generation, and agentic workflows. It is designed for complex tasks that require strong instruction following, long-context understanding, and reliable tool use.\n\n\n## API Usage\n\nYou can interact with the DeepSeek-V4-Pro model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-5 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Pro","cacheHit":false,"pricePer1mPromptToken":957000,"pricePer1mCompletionToken":1914000,"originPricePer1mCompletionToken":3480000,"originPricePer1mPromptToken":1740000,"percentageDiscount":4500,"effectiveAt":"2026-08-29T04:42:52.555486Z","endedAt":"","id":"ec8e0e46-ed9f-46cb-a7cb-6155b418b7b4","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"B200","gpuPrice":4000000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":957000,"outputPrice":1914000,"cacheReadPrice":79750,"originInputPrice":1740000,"originOutputPrice":3480000,"originCacheReadPrice":145000}]},"endpoints":[]},{"artifact_id":"de6e71be-d065-49a7-b77c-53cec002d8b2","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-13T05:27:55.340283374Z","update_at":"2026-10-17T09:28:57.850673474Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"DeepSeek-V4-Pro-0813","artifact_description":"deepseek-ai/DeepSeek-V4-Pro-0813","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/deepseek-ai/DeepSeek-V4-Pro-0813","artifact_template_id":"de6e71be-d065-49a7-b77c-53cec002d8b2","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"DeepSeek","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/deepseek-ai/DeepSeek-V4-Pro-0813","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# deepseek-ai/DeepSeek-V4-Pro-0813\n\nDeepSeek V4 Pro 0813 is a large-scale mixture-of-experts model from DeepSeek. This is the GA release of DeepSeek V4 Pro.\n\n## API Usage\n\nYou can interact with the deepseek-ai/DeepSeek-V4-Pro-0813 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V4-Pro-0813.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro-0813\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro-0813\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of an city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Hows the weather like in Qingdao today\"\n        }\n    ]\n}'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V4-Pro-0813\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"deepseek-ai/DeepSeek-V4-Pro-0813","cacheHit":false,"pricePer1mPromptToken":1056000,"pricePer1mCompletionToken":3168000,"originPricePer1mCompletionToken":3960000,"originPricePer1mPromptToken":1320000,"percentageDiscount":2000,"effectiveAt":"2026-08-18T23:46:19.840277Z","endedAt":"","id":"7b4cba53-7701-402f-9e10-01dbfff821d5","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1056000,"outputPrice":3168000,"cacheReadPrice":35200,"originInputPrice":1320000,"originOutputPrice":3960000,"originCacheReadPrice":44000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"dbe2292c-a925-41eb-ab4d-0e6eecaa1562","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-12-19T00:25:48.047301766Z","update_at":"2026-10-17T09:28:58.574972792Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini 3 Flash","artifact_description":"gemini 3 flash","artifact_tags":[],"artifact_volume_path":"models/google/gemini-3-flash-preview","artifact_template_id":"dbe2292c-a925-41eb-ab4d-0e6eecaa1562","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Google"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-3-flash-preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemini 3 Flash Preview\n\nGoogle Gemini 3 Flash Preview is a fast and efficient multimodal large language model from Google. It supports text, images, video, audio, and PDF inputs, optimized for speed and cost-effectiveness while maintaining high-quality responses.\n\n## API Usage\n\nYou can interact with the Gemini 3 Flash Preview model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 3 Flash Preview.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3-flash-preview\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n#### example for function call\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemini-3-flash-preview\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n#### example for vision (image input)\n```\n### example for vision (image input)\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3-flash-preview\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 100\n  }'\n```\n\n#### example for video input\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemini-3-flash-preview\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 1000\n  }'\n```\n\n### Gemini Native API\n#### basic generateContent\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3-flash-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ]\n  }'\n```\n\n#### example for video input (Gemini Native)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-3-flash-preview:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }]\n  }'\n```\n\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemini-3-flash-preview\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/models/gemini-3-flash-preview:generateContent\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n)\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\nvideo_metadata_part = types.Part(video_metadata=types.VideoMetadata(fps=10.0))\ntry:\n    response = client.models.generate_content(\n        model=\"google/gemini-3-flash-preview\",\n        contents=[\"Describe this video.\", video_part, video_metadata_part],\n        config=video_config\n    )\n    print(\"--- Gemini 3 Flash reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```"},"model_price":{"modelName":"google/gemini-3-flash-preview","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":3000000,"originPricePer1mCompletionToken":3000000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2025-12-18T07:38:04.98Z","endedAt":"","id":"40357633-99e7-4074-84bf-e27545f7c145","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"60b8a012-73ec-4c04-923d-8176ef91aa96","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-14T00:51:26.318440452Z","update_at":"2026-11-04T06:04:47.426684829Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemini-4-Argon","artifact_description":"Gemini-4-Argon","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/google/gemini-4-argon","artifact_template_id":"60b8a012-73ec-4c04-923d-8176ef91aa96","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Gemini","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/google/gemini-4-argon","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"\n# Google Gemini 4 Argon\n\nGoogle Gemini 4 Argon is Google's flagship reasoning model in the Gemini 4 series, designed for complex software engineering, autonomous agents, scientific and mathematical reasoning, and demanding enterprise workflows. It combines advanced multimodal understanding with deep reasoning and long-context processing for tasks involving code, text, images, video, audio, and documents.\n\nThe model supports a unified context window of up to 1,048,576 tokens, function calling, structured outputs, Google Search grounding, and configurable thinking levels (low,medium, high`). The default thinking level is medium. Thinking tokens count toward output token limits.\n\n## API Usage\n\nYou can interact with the Gemini 4 Argon model through our OpenAI-compatible APIs and Gemini native APIs. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemini 4 Argon.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"model\": \"google/gemini-4-argon\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"max_tokens\": 32768\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"model\": \"google/gemini-4-argon\",\n    \"max_tokens\": 32768,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_weather\",\n          \"description\": \"Get weather in a location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. San Francisco\"\n              }\n            },\n            \"required\": [\n              \"location\"\n            ]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather in San Francisco?\"\n      }\n    ]\n  }'\n```\n\n#### example for vision (image input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"model\": \"google/gemini-4-argon\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"What animal is in this image?\"},\n        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/220px-Cat_November_2010-1a.jpg\"}}\n      ]\n    }],\n    \"max_tokens\": 32768\n  }'\n```\n\n#### example for video input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"model\": \"google/gemini-4-argon\",\n    \"messages\": [{\n      \"role\": \"user\",\n      \"content\": [\n        {\"type\": \"text\", \"text\": \"Describe this video.\"},\n        {\"type\": \"video_url\", \"video_url\": {\"url\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"max_tokens\": 32768\n  }'\n```\n\n### Gemini Native API\n\n#### basic generateContent\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-4-argon:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"List 3 countries and their capitals.\"}]\n      }\n    ],\n    \"generationConfig\": {\n      \"maxOutputTokens\": 32768,\n      \"thinkingConfig\": {\n        \"thinkingLevel\": \"medium\"\n      }\n    }\n  }'\n```\n\n#### example for video input (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-4-argon:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"contents\": [{\n      \"role\": \"user\",\n      \"parts\": [\n        {\"text\": \"Describe this video.\"},\n        {\"fileData\": {\"mimeType\": \"video/mp4\", \"fileUri\": \"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\"}}\n      ]\n    }],\n    \"generationConfig\": {\n      \"maxOutputTokens\": 32768\n    }\n  }'\n```\n\n#### example for Grounding with Google Search (Gemini Native)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/models/gemini-4-argon:generateContent \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer \u003cGMI_API_KEY\u003e' \\\n  --data '{\n    \"contents\": [\n      {\n        \"role\": \"user\",\n        \"parts\": [{\"text\": \"Who won Euro 2024?\"}]\n      }\n    ],\n    \"tools\": [\n      {\n        \"google_search\": {}\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n}\n\npayload = {\n    \"model\": \"google/gemini-4-argon\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"max_tokens\": 32768\n}\n\nresponse = requests.post(url, headers=headers, json=payload, timeout=1200)\nresponse.raise_for_status()\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Google Vertex SDK\n\n```python\nfrom google import genai\nfrom google.genai import types\n\nclient = genai.Client(\n    vertexai=True,\n    http_options={\n        \"base_url\": \"https://api.gmi-serving.com/v1/\",\n        \"headers\": {\n            \"Authorization\": \"Bearer \u003cGMI_API_KEY\u003e\"\n        }\n    }\n)\n\nvideo_config = types.GenerateContentConfig(\n    system_instruction=\"You are a professional action analyst. Please answer in Spanish.\",\n    max_output_tokens=32768,\n    thinking_config=types.ThinkingConfig(thinking_level=\"medium\")\n)\n\nvideo_part = types.Part.from_uri(\n    file_uri=\"gs://cloud-samples-data/generative-ai/video/pixel8.mp4\",\n    mime_type=\"video/mp4\"\n)\n\ntry:\n    response = client.models.generate_content(\n        model=\"gemini-4-argon\",\n        contents=[\"Describe this video.\", video_part],\n        config=video_config\n    )\n    print(\"--- Gemini 4 Argon reply ---\")\n    print(response.text)\nexcept Exception as e:\n    print(f\"Failed: {e}\")\n```\n"},"model_price":{"modelName":"google/gemini-4-argon","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":10000000,"originPricePer1mCompletionToken":10000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-10-14T01:07:13.356491Z","endedAt":"","id":"fa8febda-b9ff-45d8-8cbb-d43d772a68fb","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":10000000,"cacheReadPrice":100000,"originInputPrice":2000000,"originOutputPrice":10000000,"originCacheReadPrice":100000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"ea6bbeb1-8aa5-4c58-bfe9-cfc98a6335d1","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/llm_proxy_server_refactored:v1.21","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-03T23:35:25.048407762Z","update_at":"2026-10-17T09:28:58.145552455Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemma 4 26B A4B IT","artifact_description":"Gemma 4 26B A4B IT\n","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemma-4-26b-a4b-it","artifact_template_id":"ea6bbeb1-8aa5-4c58-bfe9-cfc98a6335d1","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":256000,"quantization":"fp8","info_config_map":[],"provider":"Google"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/google/gemma-4-26b-a4b-it","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemma 4 26B A4B\n\nGoogle Gemma 4 26B A4B is a large language model from Google’s Gemma family. It is designed for high-quality text generation, reasoning, and conversational tasks. The model provides strong performance across a wide range of natural language applications, including chat, coding, and content generation.\n\n## API Usage\n\nYou can interact with the Gemma 4 26B A4B model through our OpenAI-compatible APIs. Below are examples showing how to use the model.\n\n\n\n## API Examples\n\nGenerate a response using the chat completions endpoint.\n\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemma 4 31B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemma-4-26b-a4b-it\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n#### example for function call\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemma-4-26b-a4b\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemma-4-26b-a4b-it\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n\n"},"model_price":{"modelName":"google/gemma-4-26b-a4b-it","cacheHit":false,"pricePer1mPromptToken":130000,"pricePer1mCompletionToken":400000,"originPricePer1mCompletionToken":400000,"originPricePer1mPromptToken":130000,"percentageDiscount":0,"effectiveAt":"2026-04-03T08:00:04.98Z","endedAt":"","id":"e7a9de97-f906-4893-810a-1619f281ff1f","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"8b6ca932-9a42-44af-8df7-d006abd4a193","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/llm_proxy_server_refactored:v1.21","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-03T23:32:50.41299845Z","update_at":"2026-10-17T09:28:58.151668185Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Gemma 4 31B IT","artifact_description":"Gemma 4 31B IT","artifact_tags":["LLM"],"artifact_volume_path":"models/google/gemma-4-31b-it","artifact_template_id":"8b6ca932-9a42-44af-8df7-d006abd4a193","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Google"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/google/gemma-4-31b-it","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Google Gemma 4 31B\n\nGoogle Gemma 4 31B is a large language model from Google’s Gemma family. It is designed for high-quality text generation, reasoning, and conversational tasks. The model provides strong performance across a wide range of natural language applications, including chat, coding, and content generation.\n\n## API Usage\n\nYou can interact with the Gemma 4 31B model through our OpenAI-compatible APIs. Below are examples showing how to use the model's API.\n\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Gemma 4 31B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"google/gemma-4-31b-it\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n#### example for function call\n```\n# example for function call\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 200,\n    \"model\": \"google/gemma-4-31b-it\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get weather in a location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. San Francisco\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather in San Francisco?\"\n        }\n    ]\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"google/gemma-4-31b\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n\n"},"model_price":{"modelName":"google/gemma-4-31b-it","cacheHit":false,"pricePer1mPromptToken":140000,"pricePer1mCompletionToken":400000,"originPricePer1mCompletionToken":400000,"originPricePer1mPromptToken":140000,"percentageDiscount":0,"effectiveAt":"2026-07-22T00:33:48.192932Z","endedAt":"","id":"efa6d1d5-7ff0-4e89-8297-960500ae77c2","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":140000,"outputPrice":400000,"cacheReadPrice":100000,"originInputPrice":140000,"originOutputPrice":400000,"originCacheReadPrice":100000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"23693705-d653-475a-8724-f07490104b87","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-07-31T21:20:46.958714173Z","update_at":"2026-10-27T06:45:31.753140145Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GLM-4.5-Air-FP8","artifact_description":"zai-org/GLM-4.5-Air-FP8","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/zai-org/GLM-4.5-Air-FP8","artifact_template_id":"23693705-d653-475a-8724-f07490104b87","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":131076,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-4.5-Air-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"40","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-4.5-Air-FP8","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-4.5-Air-FP8 --tp-size 4 --mem-fraction-static 0.7 --trust-remote-code --tool-call-parser glm45 --reasoning-parser glm45 --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --port 8000 --chat-template ./chat_template/glm-4.5.jinja","is_hidden":true}]},"artifact_details":{"model_description":"\n# GLM-4.5-Air-FP8\n\n**GLM-4.5-Air-FP8** is a **lightweight**, **high-efficiency**, and **FP8-quantized** variant of the GLM-4.5-Air model, designed to achieve **ultra-fast inference** with **minimal memory footprint** while maintaining the reasoning and conversational fluency of its parent model.  \n\n\n## API Usage\n\nThe **zai-org/GLM-4.5-Air-FP8** model can be accessed via the same REST API used by other GLM models.  \nIt supports both **general chat** and **function-calling** workflows.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the **chat/completions** endpoint for conversational generation.\n\n#### Shell\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-4.5-Air-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a concise and efficient AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Summarize the key benefits of FP8 quantization in AI models.\"}\n    ],\n    \"temperature\": 0.6,\n    \"max_tokens\": 600\n  }'\n```\n\n##### Function Calling\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-4.5-Air-FP8\",\n    \"temperature\": 0,\n    \"max_tokens\": 120,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_weather_forecast\",\n          \"description\": \"Retrieve weather forecast information for a given city.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"city\": {\n                \"type\": \"string\",\n                \"description\": \"Name of the target city, e.g., San Francisco.\"\n              }\n            },\n            \"required\": [\"city\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What’s the current weather in San Francisco?\"\n      }\n    ]\n  }'\n```\n\n#### Python SDK Usage\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-4.5-Air-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a concise and efficient AI assistant.\"},\n        {\"role\": \"user\", \"content\": \"Summarize the key benefits of FP8 quantization in AI models.\"}\n    ],\n    \"temperature\": 0.6,\n    \"max_tokens\": 600\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\n\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"zai-org/GLM-4.5-Air-FP8","cacheHit":false,"pricePer1mPromptToken":200000,"pricePer1mCompletionToken":1100000,"originPricePer1mCompletionToken":1100000,"originPricePer1mPromptToken":200000,"percentageDiscount":0,"effectiveAt":"2025-08-09T08:38:04.98Z","endedAt":"","id":"ec2a8f1c-f937-4ec4-9460-607f2f168275","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"84f25cc4-5c28-48b0-931f-72b09e5a8246","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-07-31T21:40:53.365884326Z","update_at":"2026-10-27T06:45:31.743121558Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GLM-4.5-FP8","artifact_description":"ZAI: GLM-4.5-FP8","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/zai-org/GLM-4.5-FP8","artifact_template_id":"84f25cc4-5c28-48b0-931f-72b09e5a8246","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-4.5-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-4.5-FP8","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-4.5-FP8 --tp-size 8 --mem-fraction-static 0.7 --trust-remote-code --tool-call-parser glm45 --reasoning-parser glm45 --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --port 8000 --allow-auto-truncate --chat-template ./chat_template/glm-4.5.jinja","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-4.5-FP8\n\n**GLM-4.5-FP8** is a high-efficiency variant of the GLM-4.5 large language model, designed for **ultra-fast inference** and **reduced memory consumption** through FP8 quantization.  \nThis model preserves the strong reasoning, multilingual, and generative performance of GLM-4.5 while significantly improving **deployment efficiency**, **latency**, and **serving cost**.\n\n\n## API Usage\n\nYou can access **GLM-4.5-FP8** through the same RESTful Chat Completions API used by other GLM models.  \nThe following examples demonstrate text generation and function-calling usage.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from GLM-4.5-FP8.\n\n#### Shell\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-4.5-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n  }'\n```\n\n##### Function Calling\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-4.5-FP8\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\"symbol\"]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```\n\n#### Python SDK Usage\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-4.5-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\n\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"zai-org/GLM-4.5-FP8","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":2200000,"originPricePer1mCompletionToken":2200000,"originPricePer1mPromptToken":600000,"percentageDiscount":0,"effectiveAt":"2025-08-09T08:38:04.98Z","endedAt":"","id":"864849d3-f834-440c-9980-de4ed0effea0","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"084b9cf3-d13e-4810-8ea9-15d2047d4520","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.22:ray_fix_otel_v2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-26T07:53:04.918437033Z","update_at":"2026-10-27T06:45:30.93739073Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GLM-5.3-CYBERSECURITY-H200-dev-TP8","artifact_description":"GLM-5.3-CYBERSECURITY-FP8","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/dealignai/GLM-5.3-CYBERSECURITY-FP8","artifact_template_id":"084b9cf3-d13e-4810-8ea9-15d2047d4520","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/dealignai/GLM-5.3-CYBERSECURITY-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":false,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"72","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"64","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"32","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"dealignai/GLM-5.3-CYBERSECURITY-FP8","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.21:ray_fix_otel_v2","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path dealignai/GLM-5.3-CYBERSECURITY-FP8 --served-model-name dealignai/GLM-5.3-CYBERSECURITY-FP8 --tp 8 --attn-dp-size 8 --moe-a2a-backend deepep --speculative-algorithm EAGLE --speculative-num-steps 1 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2 --mem-fraction-static 0.85 --chunked-prefill-size 32768 --reasoning-parser auto --tool-call-parser auto --enable-hierarchical-cache --hicache-ratio 2 --hicache-size 0 --hicache-write-policy write_through --port 8000 --enable-cache-report --max-running-requests 128","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-5.3-CYBERSECURITY-FP8\n\nGLM-5.3-CYBERSECURITY-FP8 is a native multimodal model from Z.ai. It is suited for efficient coding and long-horizon agent tasks. Its hybrid sparse and linear attention architecture maintains accurate long-context behavior while reducing compute overhead.\n\n\n## API Usage\n\nYou can interact with the GLM-5.3-CYBERSECURITY-FP8 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-5.3-CYBERSECURITY-FP8 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"dealignai/GLM-5.3-CYBERSECURITY-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"dealignai/GLM-5.3-CYBERSECURITY-FP8\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"79eaf030-b8df-4ac8-900f-44c609c8ad4d","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_glm_5_3_flash:ray_fix_otel_v1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-14T03:13:23.667124639Z","update_at":"2026-10-27T06:45:31.005864161Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GLM-5.3-Flash-B300-dev","artifact_description":"GLM-5.3-Flash","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/zai-org/GLM-5.3-Flash","artifact_template_id":"79eaf030-b8df-4ac8-900f-44c609c8ad4d","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.3-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"256","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"320","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"128","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-5.3-Flash","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_v12_b300_dev_v4","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-5.3-Flash --served-model-name zai-org/GLM-5.3-Flash --tp-size 2 --trust-remote-code --dsa-prefill-backend trtllm --dsa-decode-backend trtllm --kv-cache-dtype fp8_e4m3 --moe-runner-backend flashinfer_trtllm --reasoning-parser glm45 --tool-call-parser glm47 --port 8000 --enable-cache-report --max-running-requests 320","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-5.3-Flash\n\nGLM-5.3-Flash is a native multimodal model from Z.ai. It is suited for efficient coding and long-horizon agent tasks. Its hybrid sparse and linear attention architecture maintains accurate long-context behavior while reducing compute overhead.\n\n\n## API Usage\n\nYou can interact with the GLM-5.3-Flash model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-5.3-Flash model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"zai-org/GLM-5.3-Flash","cacheHit":false,"pricePer1mPromptToken":75000,"pricePer1mCompletionToken":250000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":150000,"percentageDiscount":5000,"effectiveAt":"2026-08-26T19:19:37.957197Z","endedAt":"","id":"ad910b36-0548-427c-9a68-4fbb3657098f","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"B300","gpuPrice":4700000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":75000,"outputPrice":250000,"cacheReadPrice":15000,"originInputPrice":150000,"originOutputPrice":500000,"originCacheReadPrice":30000}]},"endpoints":[]},{"artifact_id":"4d653428-1d87-40dd-ba8e-a654974174ae","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.21:ray_fix_otel_v2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-16T08:35:07.699416609Z","update_at":"2026-10-27T06:45:30.947814837Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GLM-5.3-Flash-H200-dev-TP4","artifact_description":"GLM-5.3-Flash","artifact_tags":["LLM","Text-to-Text","DeepSeek"],"artifact_volume_path":"models/zai-org/GLM-5.3-Flash","artifact_template_id":"4d653428-1d87-40dd-ba8e-a654974174ae","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.3-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":false,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"48","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"64","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-5.3-Flash","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.20:ray_fix_otel_v2","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-5.3-Flash --served-model-name zai-org/GLM-5.3-Flash --tp-size 4 --ep-size 4 --trust-remote-code --dsa-prefill-backend tilelang --dsa-decode-backend tilelang --kv-cache-dtype bfloat16 --moe-runner-backend deep_gemm --enable-hierarchical-cache --hicache-size 32 --reasoning-parser auto --tool-call-parser auto --port 8000 --enable-cache-report --max-running-requests 64 ","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-5.3-Flash\n\nGLM-5.3-Flash is a native multimodal model from Z.ai. It is suited for efficient coding and long-horizon agent tasks. Its hybrid sparse and linear attention architecture maintains accurate long-context behavior while reducing compute overhead.\n\n\n## API Usage\n\nYou can interact with the GLM-5.3-Flash model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-5.3-Flash model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-5.3-Flash\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"zai-org/GLM-5.3-Flash","cacheHit":false,"pricePer1mPromptToken":75000,"pricePer1mCompletionToken":250000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":150000,"percentageDiscount":5000,"effectiveAt":"2026-08-26T19:19:37.957197Z","endedAt":"","id":"ad910b36-0548-427c-9a68-4fbb3657098f","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":75000,"outputPrice":250000,"cacheReadPrice":15000,"originInputPrice":150000,"originOutputPrice":500000,"originCacheReadPrice":30000}]},"endpoints":[]},{"artifact_id":"6aa0142b-ca73-4262-b123-dfc4b5405a30","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-01T22:10:08.575096273Z","update_at":"2026-10-17T09:28:57.595197512Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"gpt-6.1-sol","artifact_description":"gpt-6.1-sol","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-6.1-sol","artifact_template_id":"6aa0142b-ca73-4262-b123-dfc4b5405a30","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-6.1-sol","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# openai/gpt-6.1-sol\n\nGPT-6.1-Sol is a high-performance reasoning model from OpenAI designed for complex coding, computer use, agentic workflows, and demanding professional tasks. It provides strong reasoning capabilities for workloads that require multi-step problem solving, software engineering, tool use, long-context analysis, and autonomous task execution.\n\nThe model supports a 1.05M-token context window with up to 128K output tokens and accepts both text and image inputs. GPT-6.1-Sol supports configurable reasoning through the reasoning_effort parameter, with supported values of low, medium, high, xhigh, and max. The default reasoning effort is medium. It also supports streaming, structured outputs, and function calling.\n\n## API Usage\n\nYou can interact with the openai/gpt-6.1-sol model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GPT-6.1-Sol.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6.1-sol\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"reasoning_effort\": \"medium\",\n    \"max_completion_tokens\": 500\n  }'\n```\n\n#### example for image input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6.1-sol\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this image?\"},\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://example.com/image.jpg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"reasoning_effort\": \"medium\",\n    \"max_completion_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6.1-sol\",\n    \"reasoning_effort\": \"medium\",\n    \"max_completion_tokens\": 100,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"query_weather\",\n          \"description\": \"Get weather of a city, the user should supply a city first\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"city\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. Beijing\"\n              }\n            },\n            \"required\": [\n              \"city\"\n            ]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"Hows the weather like in Qingdao today\"\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6.1-sol\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"reasoning_effort\": \"medium\",\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### example for image input\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6.1-sol\",\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://example.com/image.jpg\"\n                    }\n                }\n            ]\n        }\n    ],\n    \"reasoning_effort\": \"medium\",\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"openai/gpt-6.1-sol","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":10000000,"originPricePer1mCompletionToken":10000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-10-02T04:39:24.027392Z","endedAt":"","id":"7d96d1cd-743b-496c-977a-ed847936ce9d","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":10000000,"cacheReadPrice":100000,"cacheWritePrice":2500000,"originInputPrice":2000000,"originOutputPrice":10000000,"originCacheReadPrice":100000,"originCacheWritePrice":2500000},{"threshold":272000,"inputPrice":4000000,"outputPrice":15000000,"cacheReadPrice":200000,"cacheWritePrice":5000000,"originInputPrice":4000000,"originOutputPrice":15000000,"originCacheReadPrice":200000,"originCacheWritePrice":5000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"c384aac4-dd25-424a-a86d-591dbad9b2e0","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-05T03:14:28.048094767Z","update_at":"2026-10-17T09:28:57.723950238Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-6-Astra","artifact_description":"GPT-6-Astra","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/openai/gpt-6-astra","artifact_template_id":"c384aac4-dd25-424a-a86d-591dbad9b2e0","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[{"key":"Enabled","value":"False","type":"text","min":0,"max":0,"step":0,"display_name":"ZDR","category":"general"}],"provider":"OpenAI","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-6-astra","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI GPT-6-Astra\n\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n    - [Create a model response](#create-a-model-response)\n        - [Default (Responses)](#default-1)\n        - [Streaming (Responses)](#streaming-1)\n        - [Reasoning](#reasoning)\n        - [Functions (Responses)](#functions-1)\n        - [Image Input (Responses)](#image-input-1)\n        - [File Input](#file-input)\n\nGPT-6-Astra is OpenAI’s frontier model designed for complex professional workloads, with stronger reasoning, higher reliability, and improved efficiency on difficult tasks. It supports text and image inputs, making it suitable for large-scale reasoning, coding, multilingual conversations, and multimodal workflows.\n\n## API Usage\n\nYou can interact with the OpenAI GPT-6-Astra model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT-6-Astra.\n\n### Create chat completion\n\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-6-astra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=endpoint,\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-6-astra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=endpoint,\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Hello!\"}\n    ],\n    stream=True\n)\n\nfor chunk in completion:\n    print(chunk.choices[0].delta)\n```\n\n#### Image Input\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-6-astra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=endpoint,\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"text\",\n                    \"text\": \"What's in this image?\"\n                },\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0])\n```\n\n#### Functions\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"openai/gpt-6-astra\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=endpoint,\n    api_key=api_key\n)\n\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_current_weather\",\n            \"description\": \"Get the current weather in a given location\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"location\": {\n                        \"type\": \"string\",\n                        \"description\": \"The city and state, e.g. San Francisco, CA\",\n                    },\n                    \"unit\": {\n                        \"type\": \"string\",\n                        \"enum\": [\"celsius\", \"fahrenheit\"]\n                    },\n                },\n                \"required\": [\"location\"],\n            },\n        }\n    }\n]\n\nmessages = [\n    {\n        \"role\": \"user\",\n        \"content\": \"What's the weather like in Boston today?\"\n    }\n]\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=messages,\n    tools=tools,\n    tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\n\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6-astra\",\n    \"messages\": [\n        {\n            \"role\": \"system\",\n            \"content\": \"You are a helpful AI assistant\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"List 3 countries and their capitals.\"\n        }\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Create a model response\n\nOpenAI's most advanced interface for generating model responses. Supports text and image inputs, and text outputs. Create stateful interactions with the model, using the output of previous responses as input. Extend the model's capabilities with built-in tools for file search, web search, computer use, and more. Allow the model access to external systems and data using function calling.\n\n#### Default (Responses)\n\n```bash\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"input\": \"Tell me a three sentence bedtime story about a unicorn.\"\n  }'\n```\n\n#### Streaming (Responses)\n\n```bash\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"instructions\": \"You are a helpful assistant.\",\n    \"input\": \"Hello!\",\n    \"stream\": true\n  }'\n```\n\n#### Reasoning\n\n```bash\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"input\": \"How much wood would a woodchuck chuck?\",\n    \"reasoning\": {\n      \"effort\": \"low\"\n    }\n  }'\n```\n\n#### Functions (Responses)\n\n```bash\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"input\": \"What is the weather like in Boston today?\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"name\": \"get_current_weather\",\n        \"description\": \"Get the current weather in a given location\",\n        \"parameters\": {\n          \"type\": \"object\",\n          \"properties\": {\n            \"location\": {\n              \"type\": \"string\",\n              \"description\": \"The city and state, e.g. San Francisco, CA\"\n            },\n            \"unit\": {\n              \"type\": \"string\",\n              \"description\": \"Temperature unit\",\n              \"enum\": [\"celsius\", \"fahrenheit\"]\n            }\n          },\n          \"required\": [\"location\", \"unit\"]\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n\n#### Image Input (Responses)\n\n```bash\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"input_text\",\n            \"text\": \"what is in this image?\"\n          },\n          {\n            \"type\": \"input_image\",\n            \"image_url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n          }\n        ]\n      }\n    ]\n  }'\n```\n\n#### File Input\n\n```bash\ncurl https://api.gmi-serving.com/v1/responses \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"openai/gpt-6-astra\",\n    \"input\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"input_text\",\n            \"text\": \"what is in this file?\"\n          },\n          {\n            \"type\": \"input_file\",\n            \"file_url\": \"https://www.berkshirehathaway.com/letters/2024ltr.pdf\"\n          }\n        ]\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"openai/gpt-6-astra","cacheHit":false,"pricePer1mPromptToken":10000000,"pricePer1mCompletionToken":50000000,"originPricePer1mCompletionToken":50000000,"originPricePer1mPromptToken":10000000,"percentageDiscount":0,"effectiveAt":"2026-09-05T04:08:13.185168Z","endedAt":"","id":"45bc7451-1d43-4d89-81f7-6aefaf3ab197","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":10000000,"outputPrice":50000000,"cacheReadPrice":1000000,"cacheWritePrice":12500000,"originInputPrice":10000000,"originOutputPrice":50000000,"originCacheReadPrice":1000000,"originCacheWritePrice":12500000},{"threshold":272000,"inputPrice":20000000,"outputPrice":75000000,"cacheReadPrice":2000000,"cacheWritePrice":25000000,"originInputPrice":20000000,"originOutputPrice":75000000,"originCacheReadPrice":2000000,"originCacheWritePrice":25000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"6a7bb75b-4b97-49bc-b8a6-38dc23757075","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-23T02:47:17.636425815Z","update_at":"2026-10-17T09:28:57.66425275Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-6 Luna","artifact_description":"GPT-6 Luna","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-6-luna","artifact_template_id":"6a7bb75b-4b97-49bc-b8a6-38dc23757075","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-6-luna","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# openai/gpt-6-luna\n\nGPT-6-Luna is an efficient reasoning model from OpenAI designed for focused, high-volume workloads. It combines strong reasoning and instruction following with low latency and cost, making it well suited for applications such as classification, extraction, summarization, customer support, content processing, coding assistance, and large-scale agent workflows.\n\nThe model supports a 1.05M-token context window with up to 128K output tokens and accepts both text and image inputs. It supports configurable reasoning through the reasoning_effort parameter, with supported values of none, low, medium, high, xhigh, and max. GPT-6-Luna also supports streaming, structured outputs, and function calling.\n\n## API Usage\n\nYou can interact with the openai/gpt-6-luna model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GPT-6-Luna.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6-luna\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n  }'\n```\n\n#### example for image input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6-luna\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this image?\"},\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://example.com/image.jpg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6-luna\",\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 100,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"query_weather\",\n          \"description\": \"Get weather of a city, the user should supply a city first\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"city\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. Beijing\"\n              }\n            },\n            \"required\": [\n              \"city\"\n            ]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"Hows the weather like in Qingdao today\"\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6-luna\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### example for image input\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6-luna\",\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://example.com/image.jpg\"\n                    }\n                }\n            ]\n        }\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"openai/gpt-6-luna","cacheHit":false,"pricePer1mPromptToken":100000,"pricePer1mCompletionToken":500000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":100000,"percentageDiscount":0,"effectiveAt":"2026-09-23T23:05:24.306047Z","endedAt":"","id":"a62bea3d-9427-4eea-8e96-287fafe0ff19","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":100000,"outputPrice":500000,"cacheReadPrice":10000,"cacheWritePrice":125000,"originInputPrice":100000,"originOutputPrice":500000,"originCacheReadPrice":10000,"originCacheWritePrice":125000},{"threshold":272000,"inputPrice":200000,"outputPrice":750000,"cacheReadPrice":20000,"cacheWritePrice":250000,"originInputPrice":200000,"originOutputPrice":750000,"originCacheReadPrice":20000,"originCacheWritePrice":250000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"12b03079-038a-44f6-ac45-a8a84b6588e5","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-23T02:43:43.746043189Z","update_at":"2026-10-17T09:28:57.67061626Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT-6 Sol","artifact_description":"GPT-6 Sol","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-6-sol","artifact_template_id":"12b03079-038a-44f6-ac45-a8a84b6588e5","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"OpenAI","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-6-sol","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# openai/gpt-6-sol\n\nGPT-6-Sol is a high-performance reasoning model from OpenAI designed for complex coding, agentic workflows, and demanding professional tasks. It provides stronger reasoning capabilities for workloads that require multi-step problem solving, software engineering, tool use, long-context analysis, and autonomous task execution.\n\nThe model supports a 1.05M-token context window with up to 128K output tokens and accepts both text and image inputs. GPT-6-Sol supports configurable reasoning through the reasoning_effort parameter, with supported values of none, low, medium, high, xhigh, and max. It also supports streaming, structured outputs, and function calling.\n\n## API Usage\n\nYou can interact with the openai/gpt-6-sol model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GPT-6-Sol.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6-sol\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n  }'\n```\n\n#### example for image input\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6-sol\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this image?\"},\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://example.com/image.jpg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-6-sol\",\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 100,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"query_weather\",\n          \"description\": \"Get weather of a city, the user should supply a city first\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"city\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. Beijing\"\n              }\n            },\n            \"required\": [\n              \"city\"\n            ]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"Hows the weather like in Qingdao today\"\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6-sol\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### example for image input\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-6-sol\",\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"Describe this image.\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://example.com/image.jpg\"\n                    }\n                }\n            ]\n        }\n    ],\n    \"reasoning_effort\": \"none\",\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"openai/gpt-6-sol","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":10000000,"originPricePer1mCompletionToken":10000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-09-23T23:04:03.868204Z","endedAt":"","id":"ec7bdb3d-42d8-40fd-b602-e71f92fe364b","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":10000000,"cacheReadPrice":200000,"cacheWritePrice":2500000,"originInputPrice":2000000,"originOutputPrice":10000000,"originCacheReadPrice":200000,"originCacheWritePrice":2500000},{"threshold":272000,"inputPrice":4000000,"outputPrice":15000000,"cacheReadPrice":400000,"cacheWritePrice":5000000,"originInputPrice":4000000,"originOutputPrice":15000000,"originCacheReadPrice":400000,"originCacheWritePrice":5000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"589d6273-aca4-4ea0-93da-c81f040ca14b","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-14T14:14:46.366907004Z","update_at":"2026-10-17T09:28:58.501091644Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT OSS 120b","artifact_description":"gpt-oss-120b","artifact_tags":["LLM"],"artifact_volume_path":"models/openai/gpt-oss-120b","artifact_template_id":"589d6273-aca4-4ea0-93da-c81f040ca14b","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"OpenAI"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-oss-120b","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max completion tokens","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# OpenAI GPT OSS 120B\n\nOpenAI GPT OSS 120B is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the OpenAI GPT OSS 120B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT OSS 120B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-oss-120b\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-oss-120b\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"openai/gpt-oss-120b","cacheHit":false,"pricePer1mPromptToken":50000,"pricePer1mCompletionToken":250000,"originPricePer1mCompletionToken":250000,"originPricePer1mPromptToken":50000,"percentageDiscount":0,"effectiveAt":"2025-11-13T05:38:03.98Z","endedAt":"","id":"fced3802-cee8-434d-99c1-af58cb442ef5","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"519732eb-c1d2-4953-96ea-b261182dcc0f","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v7_gpt_oss","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-10-07T21:56:56.249256646Z","update_at":"2026-10-27T06:45:31.554079768Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"GPT OSS 20b","artifact_description":"OpenAI: GPT OSS 20b","artifact_tags":["Text-to-Text"],"artifact_volume_path":"models/openai/gpt-oss-20b","artifact_template_id":"519732eb-c1d2-4953-96ea-b261182dcc0f","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp4","info_config_map":[],"provider":"OpenAI"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/openai/gpt-oss-20b","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"24","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"openai/gpt-oss-20b","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"10000000","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path openai/gpt-oss-20b --tp-size 1 --mem-fraction-static 0.9 --trust-remote-code --port 8000 --allow-auto-truncate --chat-template ./chat_template/gpt-oss.jinja --reasoning-parser gpt-oss --tool-call-parser gpt-oss","is_hidden":true}]},"artifact_details":{"model_description":"# OpenAI GPT OSS 20B\n\nOpenAI GPT OSS 20B is a highly capable open-source large language model built with 20 billion parameters. It represents a balanced trade-off between computational efficiency and model performance, offering strong reasoning, generation, and comprehension capabilities across a wide range of tasks — from natural language understanding to structured output generation.\n\n## API Usage\n\nYou can interact with the OpenAI GPT OSS 20B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of OpenAI GPT OSS 20B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"openai/gpt-oss-20b\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"openai/gpt-oss-20b\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"openai/gpt-oss-20b","cacheHit":false,"pricePer1mPromptToken":40000,"pricePer1mCompletionToken":150000,"originPricePer1mCompletionToken":150000,"originPricePer1mPromptToken":40000,"percentageDiscount":0,"effectiveAt":"2025-10-13T08:38:04.98Z","endedAt":"","id":"628e4fff-2a75-431d-8481-cceca5c93a39","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"95a8cd9c-de97-456d-8828-261c81e42ac2","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-14T19:28:58.881922845Z","update_at":"2026-10-17T09:28:57.829190115Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Grok 4.6","artifact_description":"grok-4.6","artifact_tags":["LLM"],"artifact_volume_path":"models/xai/grok4.6","artifact_template_id":"95a8cd9c-de97-456d-8828-261c81e42ac2","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":2000000,"quantization":"fp8","info_config_map":[],"provider":"Grok"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/xai/grok4.6","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# xAI Grok 4.6\n## \n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n\nGrok 4.6 is xAI's frontier multimodal model with strong reasoning, coding, and agentic capabilities, and a large context window. Read more about the model on xAI's news post.\n\n## API Usage\n\nYou can interact with the Grok-4.6 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Grok-4.6.\n### Create chat completion\nThe Chat Completions API endpoint will generate a model response from a list of messages comprising a conversation.\n\n#### Default\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.6\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.6\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.6\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.6\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n\n```\n#### Image Input\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.6\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.6\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0])\n\n```\n#### Functions\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"x-ai/grok-4.6\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n```\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"x-ai/grok-4.6\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"x-ai/grok-4.6\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"xai/grok4.6","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":6000000,"originPricePer1mCompletionToken":6000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-08-14T19:53:21.115385Z","endedAt":"","id":"32bb7316-3073-4ac5-8a92-a07908beea2a","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":6000000,"cacheReadPrice":500000,"originInputPrice":2000000,"originOutputPrice":6000000,"originCacheReadPrice":500000},{"threshold":200000,"inputPrice":4000000,"outputPrice":12000000,"cacheReadPrice":1000000,"originInputPrice":4000000,"originOutputPrice":12000000,"originCacheReadPrice":1000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"5b75ac7b-96c8-40c6-a72a-ae62b9e68fe8","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-01T02:15:45.300086841Z","update_at":"2026-10-17T09:28:57.756416887Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"InclusionAI: Ling-3.0-flash-fin","artifact_description":"inclusionAI/Ling-3.0-flash-fin","artifact_tags":["LLM"],"artifact_volume_path":"models/inclusionAI/Ling-3.0-flash-fin","artifact_template_id":"5b75ac7b-96c8-40c6-a72a-ae62b9e68fe8","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"bf16","info_config_map":[{"key":"","value":"","type":"text","min":0,"max":0,"step":0,"display_name":"","category":"general"}],"provider":"","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/inclusionAI/Ling-3.0-flash-fin","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":1024,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Max Tokens","category":"default","description":""},{"key":"temperature","value":0.6,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# inclusionAI/Ling-3.0-flash-fin\n\nLing-3.0-flash-fin is a finance-focused mixture-of-experts model from inclusionAI, built on Ling-3.0-flash with 124B total parameters and 5.1B activated parameters per token. It is designed for real-world investment workflows that require complex multi-step tasks and long-horizon planning and execution, while retaining general capabilities in reasoning, coding, and mathematics.\n\n## API Usage\n\nYou can interact with the inclusionAI/Ling-3.0-flash-fin model through a RESTful API using an OpenAI-compatible interface. Ling-3.0-flash-fin is suitable for financial analysis assistants, investment research agents, quantitative workflow automation, and long-horizon planning tasks that combine tool use with domain-specific reasoning.\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from Ling-3.0-flash-fin.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"inclusionAI/Ling-3.0-flash-fin\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful financial analysis assistant.\"},\n      {\"role\": \"user\", \"content\": \"Summarize the key factors that affect a company'\\''s price-to-earnings ratio.\"}\n    ],\n    \"temperature\": 0.6,\n    \"top_p\": 0.95,\n    \"top_k\": 20,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Shell (Disable Thinking)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"inclusionAI/Ling-3.0-flash-fin\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"What is the difference between a bull market and a bear market?\"}\n    ],\n    \"chat_template_kwargs\": {\"enable_thinking\": false},\n    \"temperature\": 0.6,\n    \"top_p\": 0.95,\n    \"top_k\": 20,\n    \"max_tokens\": 500\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"inclusionAI/Ling-3.0-flash-fin\",\n    \"temperature\": 0.6,\n    \"top_p\": 0.95,\n    \"top_k\": 20,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"inclusionAI/Ling-3.0-flash-fin\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful financial analysis assistant.\"},\n        {\"role\": \"user\", \"content\": \"Compare the pros and cons of investing in index funds versus individual stocks.\"}\n    ],\n    \"temperature\": 0.6,\n    \"top_p\": 0.95,\n    \"top_k\": 20,\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"inclusionAI/Ling-3.0-flash-fin","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":180000,"originPricePer1mPromptToken":60000,"percentageDiscount":10000,"effectiveAt":"2026-09-02T01:48:46.35643Z","endedAt":"","id":"ecc996d9-66f6-42de-aeb7-c35486edd160","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":0,"outputPrice":0,"originInputPrice":60000,"originOutputPrice":180000,"originCacheReadPrice":12000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"9c3648a6-a809-42ad-b651-ec9a7be4676b","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.16:ray_fix_otel_v10","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-07-28T14:03:14.332449932Z","update_at":"2026-10-27T06:45:31.073401444Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Kimi-K2.7-code-v1","artifact_description":"moonshotai/Kimi-K2.7-code\n","artifact_tags":["LLM","Text-to-Text","Text-Image-to-Text"],"artifact_volume_path":"models/moonshotai/Kimi-K2.7-Code","artifact_template_id":"9c3648a6-a809-42ad-b651-ec9a7be4676b","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/moonshotai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"int4","info_config_map":[],"provider":"Moonshot"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/Kimi-K2.7-Code","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"12","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"moonshotai/Kimi-K2.7-Code","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"previous_images","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.8:ray_fix_otel_v9_kimi","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"moonshotai/Kimi-K2.7-Code","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path moonshotai/Kimi-K2.7-Code --served-model-name moonshotai/Kimi-K2.7-Code --tp-size 8 --trust-remote-code  --tool-call-parser kimi_k2 --reasoning-parser kimi_k2 --port 8000 --enable-cache-report","is_hidden":true}]},"artifact_details":{"model_description":"# Moonshotai Kimi-K2.7-code\n\nMoonshotAI: Kimi K2.7 Code is a coding-focused model in Moonshot AI's Kimi K2 family, built to complete end-to-end programming tasks reliably over long contexts. It uses a native multimodal mixture-of-experts architecture that accepts text and image input, and it always operates in a thinking mode, preserving full reasoning content across multi-turn conversations. With a 256K-token context window, it targets long-horizon coding, agentic task decomposition, and multi-turn dialogue. The model activates 32B parameters out of roughly 1T total.\n\n## API Usage\n\nYou can interact with the Moonshotai Kimi K2.7 Code model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Moonshotai Kimi K2.7 Code.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2.7-Code\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"moonshotai/Kimi-K2.7-Code\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"moonshotai/kimi-k2.7-code","cacheHit":false,"pricePer1mPromptToken":950000,"pricePer1mCompletionToken":4000000,"originPricePer1mCompletionToken":4000000,"originPricePer1mPromptToken":950000,"percentageDiscount":0,"effectiveAt":"2026-06-12T20:14:34.581104Z","endedAt":"","id":"fca1a0c7-9e76-43ae-abff-9505ec0a027e","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":950000,"outputPrice":4000000,"cacheReadPrice":190000,"originInputPrice":950000,"originOutputPrice":4000000,"originCacheReadPrice":190000}]},"endpoints":[]},{"artifact_id":"1cbe2d1c-6f43-4918-9b9d-1062b9e27cec","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v7_gpt_oss","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-12-22T11:05:07.785118988Z","update_at":"2026-10-27T06:45:31.295865256Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Llama 3.1 70B Instruct","artifact_description":"Llama-3.1-70B-Instruct","artifact_tags":["LLM","Llama3","Text-to-Text"],"artifact_volume_path":"models/meta-llama/Llama-3.3-70B-Instruct","artifact_template_id":"1cbe2d1c-6f43-4918-9b9d-1062b9e27cec","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/llama3-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/meta-llama/Llama-3.3-70B-Instruct","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"ENV","value":"prod","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"32","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"OTEL_SERVICE_NAME","value":"meta-llama/Llama-3.1-70B-Instruct","is_hidden":false},{"key":"gpu_scheduler","value":"true","is_hidden":false},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":false},{"key":"SERVED_MODEL_NAME","value":"meta-llama/Llama-3.1-70B-Instruct","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"24","is_hidden":false},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-70B-Instruct --tp-size 2 --mem-fraction-static 0.9 --trust-remote-code --context-length 131072 --tool-call-parser llama3 --enable-metrics --port 8000 --served-model-name meta-llama/Llama-3.1-70B-Instruct --grammar-backend xgrammar --max-running-requests 32 --max-queued-requests 32","is_hidden":true}]},"artifact_details":{"model_description":"# Llama-3.1-70B-Instruct\n\nLlama-3.1-70B-Instruct is a powerful large language model while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Llama-3.1-70B-Instruct model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Llama-3.1-70B-Instruct.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"meta-llama/Llama-3.1-70B-Instruct\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"meta-llama/Llama-3.1-70B-Instruct\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"b3ec7023-e963-471e-91b1-d5ab9f1f100b","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.0:v1.3.7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-04-03T20:55:26.053025617Z","update_at":"2026-10-27T06:45:31.872819944Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Llama-3.1-8B-Instruct","artifact_description":"Llama3.1 8B Insturct","artifact_tags":["LLM","Llama3"],"artifact_volume_path":"models/meta-llama/Llama-3.1-8B-Instruct","artifact_template_id":"b3ec7023-e963-471e-91b1-d5ab9f1f100b","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/llama3-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131076,"quantization":"bf16","info_config_map":null,"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/meta-llama/Llama-3.1-8B-Instruct","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --tp-size 1 --mem-fraction-static 0.9 --trust-remote-code --context-length 131072 --tool-call-parser llama3 --allow-auto-truncate --port 8000","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false}]},"artifact_details":{"model_description":"## Model Information\n\nThe Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out). The Llama 3.1 instruction tuned text only models (8B, 70B, 405B) are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common industry benchmarks.\n\n**Model developer**: Meta\n\n**Model Architecture:** Llama 3.1 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. \n\n**Supported languages:** English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.\n\n**Llama 3.1 family of models**. Token counts refer to pretraining data only. All model versions use Grouped-Query Attention (GQA) for improved inference scalability.\n\n**Model Release Date:** July 23, 2024.\n\n**Status:** This is a static model trained on an offline dataset. Future versions of the tuned models will be released as we improve model safety with community feedback.\n\n**License:** A custom commercial license, the Llama 3.1 Community License, is available at: [https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)\n\nWhere to send questions or comments about the model Instructions on how to provide feedback or comments on the model can be found in the model [README](https://github.com/meta-llama/llama3). For more technical information about generation parameters and recipes for how to use Llama 3.1 in applications, please go [here](https://github.com/meta-llama/llama-recipes). \n\n\n## Intended Use\n\n**Intended Use Cases** Llama 3.1 is intended for commercial and research use in multiple languages. Instruction tuned text only models are intended for assistant-like chat, whereas pretrained models can be adapted for a variety of natural language generation tasks. The Llama 3.1 model collection also supports the ability to leverage the outputs of its models to improve other models including synthetic data generation and distillation. The Llama 3.1 Community License allows for these use cases. \n\n**Out-of-scope** Use in any manner that violates applicable laws or regulations (including trade compliance laws). Use in any other way that is prohibited by the Acceptable Use Policy and Llama 3.1 Community License. Use in languages beyond those explicitly referenced as supported in this model card**.\n**\u003cspan style=\"text-decoration:underline;\"\u003eNote\u003c/span\u003e: Llama 3.1 has been trained on a broader collection of languages than the 8 supported languages. Developers may fine-tune Llama 3.1 models for languages beyond the 8 supported languages provided they comply with the Llama 3.1 Community License and the Acceptable Use Policy and in such cases are responsible for ensuring that any uses of Llama 3.1 in additional languages is done in a safe and responsible manner.\n\n## How to use\n\nThis repository contains two versions of Meta-Llama-3.1-8B-Instruct, for use with transformers and with the original `llama` codebase.\n\n### Use with transformers\n\nStarting with `transformers \u003e= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function.\n\nMake sure to update your transformers installation via `pip install --upgrade transformers`.\n\n```python\nimport transformers\nimport torch\nmodel_id = \"meta-llama/Meta-Llama-3.1-8B-Instruct\"\npipeline = transformers.pipeline(\n    \"text-generation\",\n    model=model_id,\n    model_kwargs={\"torch_dtype\": torch.bfloat16},\n    device_map=\"auto\",\n)\nmessages = [\n    {\"role\": \"system\", \"content\": \"You are a pirate chatbot who always responds in pirate speak!\"},\n    {\"role\": \"user\", \"content\": \"Who are you?\"},\n]\noutputs = pipeline(\n    messages,\n    max_new_tokens=256,\n)\nprint(outputs[0][\"generated_text\"][-1])\n```\n\nNote: You can also find detailed recipes on how to use the model locally, with `torch.compile()`, assisted generations, quantised and more at [`huggingface-llama-recipes`](https://github.com/huggingface/huggingface-llama-recipes)\n\n### Tool use with transformers\n\nLLaMA-3.1 supports multiple tool use formats. You can see a full guide to prompt formatting [here](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/).\n\nTool use is also supported through [chat templates](https://huggingface.co/docs/transformers/main/chat_templating#advanced-tool-use--function-calling) in Transformers. \nHere is a quick example showing a single simple tool:\n\n```python\n# First, define a tool\ndef get_current_temperature(location: str) -\u003e float:\n    \"\"\"\n    Get the current temperature at a location.\n    \n    Args:\n        location: The location to get the temperature for, in the format \"City, Country\"\n    Returns:\n        The current temperature at the specified location in the specified units, as a float.\n    \"\"\"\n    return 22.  # A real function should probably actually get the temperature!\n# Next, create a chat and apply the chat template\nmessages = [\n  {\"role\": \"system\", \"content\": \"You are a bot that responds to weather queries.\"},\n  {\"role\": \"user\", \"content\": \"Hey, what's the temperature in Paris right now?\"}\n]\ninputs = tokenizer.apply_chat_template(messages, tools=[get_current_temperature], add_generation_prompt=True)\n```\n\nYou can then generate text from this input as normal. If the model generates a tool call, you should add it to the chat like so:\n\n```python\ntool_call = {\"name\": \"get_current_temperature\", \"arguments\": {\"location\": \"Paris, France\"}}\nmessages.append({\"role\": \"assistant\", \"tool_calls\": [{\"type\": \"function\", \"function\": tool_call}]})\n```\n\nand then call the tool and append the result, with the `tool` role, like so:\n\n```python\nmessages.append({\"role\": \"tool\", \"name\": \"get_current_temperature\", \"content\": \"22.0\"})\n```\n\nAfter that, you can `generate()` again to let the model use the tool result in the chat. Note that this was a very brief introduction to tool calling - for more information,\nsee the [LLaMA prompt format docs](https://llama.meta.com/docs/model-cards-and-prompt-formats/llama3_1/) and the Transformers [tool use documentation](https://huggingface.co/docs/transformers/main/chat_templating#advanced-tool-use--function-calling).\n\n\n### Use with `llama`\n\nPlease, follow the instructions in the [repository](https://github.com/meta-llama/llama)\n\nTo download Original checkpoints, see the example command below leveraging `huggingface-cli`:\n\n```\nhuggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include \"original/*\" --local-dir Meta-Llama-3.1-8B-Instruct\n```\n\n## Hardware and Software\n\n**Training Factors** We used custom training libraries, Meta's custom built GPU cluster, and production infrastructure for pretraining. Fine-tuning, annotation, and evaluation were also performed on production infrastructure.\n\n## Training Data\n\n**Overview:** Llama 3.1 was pretrained on ~15 trillion tokens of data from publicly available sources. The fine-tuning data includes publicly available instruction datasets, as well as over 25M synthetically generated examples. \n\n**Data Freshness:** The pretraining data has a cutoff of December 2023.\n\n## Responsibility \u0026 Safety\nAs part of our Responsible release approach, we followed a three-pronged strategy to managing trust \u0026 safety risks:\n* Enable developers to deploy helpful, safe and flexible experiences for their target audience and for the use cases supported by Llama. \n* Protect developers against adversarial users aiming to exploit Llama capabilities to potentially cause harm.\n* Provide protections for the community to help prevent the misuse of our models.\n### Responsible deployment \nLlama is a foundational technology designed to be used in a variety of use cases, examples on how Meta’s Llama models have been responsibly deployed can be found in our [Community Stories webpage](https://llama.meta.com/community-stories/). Our approach is to build the most helpful models enabling the world to benefit from the technology power, by aligning our model safety for the generic use cases addressing a standard set of harms. Developers are then in the driver seat to tailor safety for their use case, defining their own policy and deploying the models with the necessary safeguards in their Llama systems. Llama 3.1 was developed following the best practices outlined in our Responsible Use Guide, you can refer to the [Responsible Use Guide](https://llama.meta.com/responsible-use-guide/) to learn more. \n#### Llama 3.1 instruct \nOur main objectives for conducting safety fine-tuning are to provide the research community with a valuable resource for studying the robustness of safety fine-tuning, as well as to offer developers a readily available, safe, and powerful model for various applications to reduce the developer workload to deploy safe AI systems. For more details on the safety mitigations implemented please read the Llama 3 paper. \n##### **Fine-tuning data**\nWe employ a multi-faceted approach to data collection, combining human-generated data from our vendors with synthetic data to mitigate potential safety risks. We’ve developed many large language model (LLM)-based classifiers that enable us to thoughtfully select high-quality prompts and responses, enhancing data quality control. \n##### **Refusals and Tone**\nBuilding on the work we started with Llama 3, we put a great emphasis on model refusals to benign prompts as well as refusal tone. We included both borderline and adversarial prompts in our safety data strategy, and modified our safety data responses to follow  tone guidelines. \n#### Llama 3.1 systems\n**Large language models, including Llama 3.1, are not designed to be deployed in isolation but instead should be deployed as part of an overall AI system with additional safety guardrails as required.** Developers are expected to deploy system safeguards when building agentic systems. Safeguards are key to achieve the right helpfulness-safety alignment as well as mitigating safety and security risks inherent to the system and any integration of the model or system with external tools. \n\nAs part of our responsible release approach, we provide the community with [safeguards](https://llama.meta.com/trust-and-safety/) that developers should deploy with Llama models or other LLMs, including Llama Guard 3, Prompt Guard and Code Shield. All our [reference implementations](https://github.com/meta-llama/llama-agentic-system) demos contain these safeguards by default so developers can benefit from system-level safety out-of-the-box. \n#### New capabilities \nNote that this release introduces new capabilities, including a longer context window, multilingual inputs and outputs and possible integrations by developers with third party tools. Building with these new capabilities requires specific considerations in addition to the best practices that generally apply across all Generative AI use cases.\n##### **Tool-use**: \nJust like in standard software development, developers are responsible for the integration of the LLM with the tools and services of their choice. They should define a clear policy for their use case and assess the integrity of the third party services they use to be aware of the safety and security limitations when using this capability. Refer to the Responsible Use Guide for best practices on the safe deployment of the third party safeguards. \n##### **Multilinguality**: \nLlama 3.1 supports 7 languages in addition to English: French, German, Hindi, Italian, Portuguese, Spanish, and Thai. Llama may be able to output text in other languages than those that meet performance thresholds for safety and helpfulness. We strongly discourage developers from using this model to converse in non-supported languages without implementing finetuning and system controls in alignment with their policies and the best practices shared in the Responsible Use Guide. \n### Evaluations\nWe evaluated Llama models for common use cases as well as specific capabilities. Common use cases evaluations measure safety risks of systems for most commonly built applications including chat bot, coding assistant, tool calls. We built dedicated, adversarial evaluation datasets and evaluated systems composed of Llama models and Llama Guard 3 to filter input prompt and output response. It is important to evaluate applications in context, and we recommend building dedicated evaluation dataset for your use case. Prompt Guard and Code Shield are also available if relevant to the application. \n\nCapability evaluations measure vulnerabilities of Llama models inherent to specific capabilities, for which were crafted dedicated benchmarks including long context, multilingual, tools calls, coding or memorization.\n### **Red teaming**\nFor both scenarios, we conducted recurring red teaming exercises with the goal of discovering risks via adversarial prompting and we used the learnings to improve our benchmarks and safety tuning datasets. \n\nWe partnered early with subject-matter experts in critical risk areas to understand the nature of these real-world harms and how such models may lead to unintended harm for society. Based on these conversations, we derived a set of adversarial goals for the red team to attempt to achieve, such as extracting harmful information or reprogramming the model to act in a potentially harmful capacity.  The red team consisted of experts in cybersecurity, adversarial machine learning, responsible AI, and integrity in addition to multilingual content specialists with background in integrity issues in specific geographic markets.\n### Critical and other risks \nWe specifically focused our efforts on mitigating the following critical risk areas:\n#### **1- CBRNE (Chemical, Biological, Radiological, Nuclear, and Explosive materials) helpfulness**\nTo assess risks related to proliferation of chemical and biological weapons, we performed uplift testing designed to assess whether use of Llama 3.1 models could meaningfully increase the capabilities of malicious actors to plan or carry out attacks using these types of weapons. \n#### **2. Child Safety**\nChild Safety risk assessments were conducted using a team of experts, to assess the model’s capability to produce outputs that could result in Child Safety risks and inform on any necessary and appropriate risk mitigations via fine tuning. We leveraged those expert red teaming sessions to expand the coverage of our evaluation benchmarks through Llama 3 model development.  For Llama 3, we conducted new in-depth sessions using objective based methodologies to assess the model risks along multiple attack vectors including the additional languages Llama 3 is trained on. We also partnered with content specialists to perform red teaming exercises assessing potentially violating content while taking account of market specific nuances or experiences. \n#### **3. Cyber attack enablement**\nOur cyber attack uplift study investigated whether LLMs can enhance human capabilities in hacking tasks, both in terms of skill level and speed.\n\nOur attack automation study focused on evaluating the capabilities of LLMs when used as autonomous agents in cyber offensive operations, specifically in the context of ransomware attacks. This evaluation was distinct from previous studies that considered LLMs as interactive assistants. The primary objective was to assess whether these models could effectively function as independent agents in executing complex cyber-attacks without human intervention.\n\nOur study of Llama-3.1-405B’s social engineering uplift for cyber attackers was conducted to assess the effectiveness of AI models in aiding cyber threat actors in spear phishing campaigns. Please read our Llama 3.1 Cyber security whitepaper to learn more.\n### Community \nGenerative AI safety requires expertise and tooling, and we believe in the strength of the open community to accelerate its progress. We are active members of open consortiums, including the AI Alliance, Partnership on AI and MLCommons, actively contributing to safety standardization and transparency. We encourage the community to adopt taxonomies like the MLCommons Proof of Concept evaluation to facilitate collaboration and transparency on safety and content evaluations. Our Purple Llama tools are open sourced for the community to use and widely distributed across ecosystem partners including cloud service providers. We encourage community contributions to our [Github repository](https://github.com/meta-llama/PurpleLlama). \n\nWe also set up the [Llama Impact Grants](https://llama.meta.com/llama-impact-grants/) program to identify and support the most compelling applications of Meta’s Llama model for societal benefit across three categories: education, climate and open innovation. The 20 finalists from the hundreds of applications can be found [here](https://llama.meta.com/llama-impact-grants/#finalists). \n\nFinally, we put in place a set of resources including an [output reporting mechanism](https://developers.facebook.com/llama_output_feedback) and [bug bounty program](https://www.facebook.com/whitehat) to continuously improve the Llama technology with the help of the community.\n## Ethical Considerations and Limitations\nThe core values of Llama 3.1 are openness, inclusivity and helpfulness. It is meant to serve everyone, and to work for a wide range of use cases. It is thus designed to be accessible to people across many different backgrounds, experiences and perspectives. Llama 3.1 addresses users and their needs as they are, without insertion unnecessary judgment or normativity, while reflecting the understanding that even content that may appear problematic in some cases can serve valuable purposes in others. It respects the dignity and autonomy of all users, especially in terms of the values of free thought and expression that power innovation and progress. \n\nBut Llama 3.1 is a new technology, and like any new technology, there are risks associated with its use. Testing conducted to date has not covered, nor could it cover, all scenarios. For these reasons, as with all LLMs, Llama 3.1’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of Llama 3.1 models, developers should perform safety testing and tuning tailored to their specific applications of the model. Please refer to available resources including our [Responsible Use Guide](https://llama.meta.com/responsible-use-guide), [Trust and Safety](https://llama.meta.com/trust-and-safety/) solutions, and other [resources](https://llama.meta.com/docs/get-started/) to learn more about responsible development. \n"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"d9755097-53db-4a09-bf63-f8150f4c39fb","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v7_gpt_oss","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-20T00:55:41.56187373Z","update_at":"2026-10-27T06:45:31.403970487Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Llama 3.3 70B Instruct","artifact_description":"Llama-3.3-70B-Instruct","artifact_tags":["LLM","Llama3","Text-to-Text"],"artifact_volume_path":"models/meta-llama/Llama-3.3-70B-Instruct","artifact_template_id":"d9755097-53db-4a09-bf63-f8150f4c39fb","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/llama3-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"bf16","info_config_map":[],"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/meta-llama/Llama-3.3-70B-Instruct","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"ENV","value":"prod","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"32","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"OTEL_SERVICE_NAME","value":"meta-llama/Llama-3.3-70B-Instruct","is_hidden":false},{"key":"gpu_scheduler","value":"true","is_hidden":false},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":false},{"key":"SERVED_MODEL_NAME","value":"meta-llama/Llama-3.3-70B-Instruct","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"24","is_hidden":false},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path meta-llama/Llama-3.3-70B-Instruct --tp-size 2 --mem-fraction-static 0.9 --trust-remote-code --context-length 131072 --tool-call-parser llama3 --enable-metrics --port 8000 --served-model-name meta-llama/Llama-3.3-70B-Instruct --grammar-backend xgrammar --max-running-requests 32 --max-queued-requests 32","is_hidden":true}]},"artifact_details":{"model_description":"# Llama-3.3-70B-Instruct\n\nLlama-3.3-70B-Instruct is a powerful large language model while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Llama-3.3-70B-Instruct model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Llama-3.3-70B-Instruct.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"meta-llama/Llama-3.3-70B-Instruct\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"meta-llama/Llama-3.3-70B-Instruct\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"meta-llama/Llama-3.3-70B-Instruct","cacheHit":false,"pricePer1mPromptToken":250000,"pricePer1mCompletionToken":750000,"originPricePer1mCompletionToken":750000,"originPricePer1mPromptToken":250000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:55.164798Z","endedAt":"","id":"9043c738-3efb-4e71-83cf-c9e64377ce1d","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"2cf0dd66-225d-4798-99d0-fe9b46b7a957","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.1:v1.3.9","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-04-26T20:07:58.702843068Z","update_at":"2026-10-27T06:45:31.85098233Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Llama-4 Maverick 17B 128E Instruct FP8","artifact_description":"Llama-4-Maverick-17B-128E-Instruct FP8","artifact_tags":["LLM","Llama4","Text-Image-to-Text","Text-to-Text"],"artifact_volume_path":"models/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8","artifact_template_id":"2cf0dd66-225d-4798-99d0-fe9b46b7a957","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/llama3-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"CHAT_TEMPLATE","value":"llama-4","is_hidden":false},{"key":"TOOL_CALL_PARSER","value":"llama3","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"REDIS_HOST","value":"ie-redis-service.gmi-inference-engine.svc.cluster.local","is_hidden":false},{"key":"CONTEXT_LENGTH","value":"1048576","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"20","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.6","is_hidden":false},{"key":"OTEL_SERVICE_NAME","value":"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 --tp-size 8 --mem-fraction-static 0.6 --trust-remote-code --tool-call-parser pythonic --chat-template llama-4 --enable-metrics --enable-multimodal --served-model-name meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 --allow-auto-truncate --port 8000 --attention-backend fa3 --speculative-num-steps 2 --speculative-eagle-topk 1 --speculative-num-draft-tokens 2 --chat-template ./chat_template/tool_chat_template_llama4_pythonic.jinja","is_hidden":true}]},"artifact_details":{"model_description":"# Llama-4-Maverick-17B-128E-Instruct\n\nLlama-4-Maverick-17B-128E-Instruct is a powerful large language model maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Llama-4-Maverick-17B-128E-Instruct model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Llama-4-Maverick-17B-128E-Instruct.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8","cacheHit":false,"pricePer1mPromptToken":250000,"pricePer1mCompletionToken":800000,"originPricePer1mCompletionToken":800000,"originPricePer1mPromptToken":250000,"percentageDiscount":0,"effectiveAt":"2025-05-24T03:17:54.913Z","endedAt":"","id":"23cc7608-4923-4533-bacd-4aad1f9ea380","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"169f3961-e31c-4829-8008-2a3a05184fa8","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-04-09T19:22:01.975321517Z","update_at":"2026-10-27T06:45:31.865570796Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Llama-4 Scout 17B 16E Instruct","artifact_description":"Llama-4-Scout-17B-16E-Instruct","artifact_tags":["LLM","Llama4","Text-Image-to-Text","Text-to-Text"],"artifact_volume_path":"models/meta-llama/Llama-4-Scout-17B-16E-Instruct","artifact_template_id":"169f3961-e31c-4829-8008-2a3a05184fa8","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/llama3-ai-icon.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"bf16","info_config_map":[],"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/meta-llama/Llama-4-Scout-17B-16E-Instruct","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":false},{"key":"CHAT_TEMPLATE","value":"llama-4","is_hidden":false},{"key":"TOOL_CALL_PARSER","value":"llama3","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"REDIS_HOST","value":"ie-redis-service.gmi-inference-engine.svc.cluster.local","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"20","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"20","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"OTEL_SERVICE_NAME","value":"meta-llama/Llama-4-Scout-17B-16E-Instruct","is_hidden":false},{"key":"gpu_scheduler","value":"true","is_hidden":false},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path meta-llama/Llama-4-Scout-17B-16E-Instruct --tp-size 8 --mem-fraction-static 0.82 --trust-remote-code --context-length 1048576 --tool-call-parser llama3 --enable-metrics --enable-multimodal --port 8000 --chat-template ./chat_template/tool_chat_template_llama4_pythonic.jinja  --attention-backend fa3","is_hidden":true}]},"artifact_details":{"model_description":"# Llama-4-Scout-17B-16E-Instruct\n\nLlama-4-Scout-17B-16E-Instruct is a powerful large language model maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Llama-4-Scout-17B-16E-Instruct model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Llama-4-Scout-17B-16E-Instruct.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"meta-llama/Llama-4-Scout-17B-16E-Instruct\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"meta-llama/Llama-4-Scout-17B-16E-Instruct\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"meta-llama/Llama-4-Scout-17B-16E-Instruct","cacheHit":false,"pricePer1mPromptToken":80000,"pricePer1mCompletionToken":500000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":80000,"percentageDiscount":0,"effectiveAt":"2025-04-17T03:17:54.653794Z","endedAt":"","id":"c2938a07-44ca-4ff5-819d-075cba7940a3","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"302d1538-75be-4c10-81a0-7e9037056e59","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-29T19:58:29.08539622Z","update_at":"2026-10-17T09:28:58.079767107Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiMo-V2.5","artifact_description":"MiMo-V2.5","artifact_tags":["LLM"],"artifact_volume_path":"models/XiaomiMiMo/MiMo-V2.5","artifact_template_id":"302d1538-75be-4c10-81a0-7e9037056e59","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/xiaomimimo2.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"fp8","info_config_map":[],"provider":"XiaomiMiMo"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/XiaomiMiMo/MiMo-V2.5","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Xiaomi MiMo-V2.5\n##\n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n    - [Create chat completion](#create-chat-completion)\n        - [Default](#default)\n        - [Streaming](#streaming)\n        - [Image Input](#image-input)\n        - [Functions](#functions)\n        - [Python](#python)\n\nMiMo-V2.5 is Xiaomi’s **multimodal** MiMo generation: it combines a **sparse MoE** language backbone (~310B total parameters, ~15B active) trained on **~48T tokens** with **in-house visual and audio encoders** and projectors, so it can reason across **text, image, and audio** in one model (see the [MiMo-V2.5 announcement](https://mimo.xiaomi.com/mimo-v2-5)). Context extends up to **about 1M tokens**. On agentic and coding-style workloads it is positioned as **high capability with favorable cost versus MiMo-V2.5-Pro** on Xiaomi billing (Token Plan rates are documented on the same page). Use API model id `mimo-v2.5` on Xiaomi’s platform; on this gateway the routed model name is `XiaomiMiMo/MiMo-V2.5`.\n\n## API Usage\n\nYou can interact with MiMo-V2.5 through standard OpenAI-compatible HTTP APIs. Below are examples using the chat completions endpoint.\n\n**Message roles:** Examples use OpenAI’s `developer` role where shown; if your vendor rejects it, use `system` instead.\n\n## API Examples\n\nGenerate a model response using the chat completions endpoint of MiMo-V2.5.\n\n### Create chat completion\n\nThe Chat Completions API generates a model reply from a list of conversation messages.\n\n#### Default\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=[\n    {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n    {\"role\": \"user\", \"content\": \"Hello!\"}\n  ],\n  stream=True\n)\n\nfor chunk in completion:\n  print(chunk.choices[0].delta)\n```\n\n#### Image Input\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Functions\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.5\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.5\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n  {\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_current_weather\",\n      \"description\": \"Get the current weather in a given location\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"The city and state, e.g. San Francisco, CA\",\n          },\n          \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n        },\n        \"required\": [\"location\"],\n      },\n    }\n  }\n]\nmessages = [{\"role\": \"user\", \"content\": \"What's the weather like in Boston today?\"}]\ncompletion = client.chat.completions.create(\n  model=model_name,\n  messages=messages,\n  tools=tools,\n  tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"XiaomiMiMo/MiMo-V2.5\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n"},"model_price":{"modelName":"XiaomiMiMo/MiMo-V2.5","cacheHit":false,"pricePer1mPromptToken":119000,"pricePer1mCompletionToken":238000,"originPricePer1mCompletionToken":280000,"originPricePer1mPromptToken":140000,"percentageDiscount":1500,"effectiveAt":"2026-07-15T23:17:45.590688Z","endedAt":"","id":"d3998f8d-9154-48aa-b9bc-9234975a1d0a","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":119000,"outputPrice":238000,"cacheReadPrice":2380,"originInputPrice":140000,"originOutputPrice":280000,"originCacheReadPrice":2800}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"ecc2d2ea-a257-4da7-895b-828aaa45b370","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-28T19:39:41.176128605Z","update_at":"2026-10-17T09:28:57.636551561Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiMo-V2.6-Flash","artifact_description":"MiMo-V2.6-Flash","artifact_tags":["LLM"],"artifact_volume_path":"models/XiaomiMiMo/MiMo-V2.6-Flash","artifact_template_id":"ecc2d2ea-a257-4da7-895b-828aaa45b370","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/xiaomimimo2.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"bf16","info_config_map":[],"provider":"XiaomiMiMo","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/XiaomiMiMo/MiMo-V2.6-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Xiaomi MiMo-V2.6-Flash\n\n##\n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n   - [Create chat completion](#create-chat-completion)\n       - [Default](#default)\n       - [Streaming](#streaming)\n       - [Image Input](#image-input)\n       - [Functions](#functions)\n       - [Python](#python)\n\nMiMo-V2.6-Flash is Xiaomi’s flagship model in the MiMo-V2.6 series, built for **advanced reasoning**, **agentic workflows**, **complex software engineering**, and **long-horizon tasks**. The V2.6 generation extends MiMo’s agent-focused architecture with stronger general intelligence and multimodal capabilities, while supporting an **up to 1M-token context window** for large codebases, documents, and extended agent workflows.\n\nMiMo-V2.6-Flash is designed for demanding workloads that require multi-step reasoning, reliable tool use, coding, and sustained execution across long contexts. It is particularly well suited for software engineering agents, research workflows, autonomous task execution, and applications that combine reasoning with multimodal understanding.\n\nUse API model id `mimo-v2.6-Flash` on Xiaomi’s platform; on this gateway the routed model name is `XiaomiMiMo/MiMo-V2.6-Flash`.\n\n## API Usage\n\nYou can interact with MiMo-V2.6-Flash through standard OpenAI-compatible HTTP APIs. Below are examples using the chat completions endpoint.\n\n**Message roles:** Examples use OpenAI’s `developer` role where shown; if your vendor rejects it, use `system` instead.\n\n## API Examples\n\nGenerate a model response using the chat completions endpoint of MiMo-V2.6-Flash.\n\n### Create chat completion\n\nThe Chat Completions API generates a model reply from a list of conversation messages.\n\n#### Default\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Flash\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Flash\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Flash\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Flash\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Hello!\"}\n    ],\n    stream=True\n)\n\nfor chunk in completion:\n    print(chunk.choices[0].delta)\n```\n\n#### Image Input\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Flash\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Flash\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Functions\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Flash\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Flash\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_current_weather\",\n            \"description\": \"Get the current weather in a given location\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"location\": {\n                        \"type\": \"string\",\n                        \"description\": \"The city and state, e.g. San Francisco, CA\",\n                    },\n                    \"unit\": {\n                        \"type\": \"string\",\n                        \"enum\": [\"celsius\", \"fahrenheit\"]\n                    },\n                },\n                \"required\": [\"location\"],\n            },\n        }\n    }\n]\n\nmessages = [\n    {\n        \"role\": \"user\",\n        \"content\": \"What's the weather like in Boston today?\"\n    }\n]\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=messages,\n    tools=tools,\n    tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\n\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Flash\",\n    \"messages\": [\n        {\n            \"role\": \"system\",\n            \"content\": \"You are a helpful AI assistant\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"List 3 countries and their capitals.\"\n        }\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\n\nprint(json.dumps(response.json(), indent=2))\n```\n````"},"model_price":{"modelName":"XiaomiMiMo/MiMo-V2.6-Flash","cacheHit":false,"pricePer1mPromptToken":112000,"pricePer1mCompletionToken":224000,"originPricePer1mCompletionToken":280000,"originPricePer1mPromptToken":140000,"percentageDiscount":2000,"effectiveAt":"2026-09-28T19:43:59.24048Z","endedAt":"","id":"a3d4c40e-5c68-4973-9d7b-766c5cbddffb","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":112000,"outputPrice":224000,"cacheReadPrice":2240,"originInputPrice":140000,"originOutputPrice":280000,"originCacheReadPrice":2800}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"3a6f9a3e-c703-4abd-af1c-42b7793de789","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-25T19:56:44.481584726Z","update_at":"2026-10-17T09:28:57.643379171Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiMo-V2.6-Pro","artifact_description":"MiMo-V2.6-Pro","artifact_tags":["LLM"],"artifact_volume_path":"models/XiaomiMiMo/MiMo-V2.6-Pro","artifact_template_id":"3a6f9a3e-c703-4abd-af1c-42b7793de789","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/xiaomimimo2.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1050000,"quantization":"bf16","info_config_map":[],"provider":"XiaomiMiMo","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/XiaomiMiMo/MiMo-V2.6-Pro","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Xiaomi MiMo-V2.6-Pro\n\n##\n- [API Usage](#api-usage)\n- [API Examples](#api-examples)\n   - [Create chat completion](#create-chat-completion)\n       - [Default](#default)\n       - [Streaming](#streaming)\n       - [Image Input](#image-input)\n       - [Functions](#functions)\n       - [Python](#python)\n\nMiMo-V2.6-Pro is Xiaomi’s flagship model in the MiMo-V2.6 series, built for **advanced reasoning**, **agentic workflows**, **complex software engineering**, and **long-horizon tasks**. The V2.6 generation extends MiMo’s agent-focused architecture with stronger general intelligence and multimodal capabilities, while supporting an **up to 1M-token context window** for large codebases, documents, and extended agent workflows.\n\nMiMo-V2.6-Pro is designed for demanding workloads that require multi-step reasoning, reliable tool use, coding, and sustained execution across long contexts. It is particularly well suited for software engineering agents, research workflows, autonomous task execution, and applications that combine reasoning with multimodal understanding.\n\nUse API model id `mimo-v2.6-pro` on Xiaomi’s platform; on this gateway the routed model name is `XiaomiMiMo/MiMo-V2.6-Pro`.\n\n## API Usage\n\nYou can interact with MiMo-V2.6-Pro through standard OpenAI-compatible HTTP APIs. Below are examples using the chat completions endpoint.\n\n**Message roles:** Examples use OpenAI’s `developer` role where shown; if your vendor rejects it, use `system` instead.\n\n## API Examples\n\nGenerate a model response using the chat completions endpoint of MiMo-V2.6-Pro.\n\n### Create chat completion\n\nThe Chat Completions API generates a model reply from a list of conversation messages.\n\n#### Default\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ]\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the capital of France?\",\n        }\n    ],\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Streaming\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"developer\",\n        \"content\": \"You are a helpful assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Hello!\"\n      }\n    ],\n    \"stream\": true\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\"role\": \"developer\", \"content\": \"You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": \"Hello!\"}\n    ],\n    stream=True\n)\n\nfor chunk in completion:\n    print(chunk.choices[0].delta)\n```\n\n#### Image Input\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\"\n            }\n          }\n        ]\n      }\n    ],\n    \"max_completion_tokens\": 300\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg\",\n                    }\n                },\n            ],\n        }\n    ],\n    max_completion_tokens=300,\n)\n\nprint(completion.choices[0].message)\n```\n\n#### Functions\n\n```bash\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer $GMI_API_KEY\" \\\n  -d '{\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Pro\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the weather like in Boston today?\"\n      }\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_current_weather\",\n          \"description\": \"Get the current weather in a given location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city and state, e.g. San Francisco, CA\"\n              },\n              \"unit\": {\n                \"type\": \"string\",\n                \"enum\": [\"celsius\", \"fahrenheit\"]\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"tool_choice\": \"auto\"\n  }'\n```\n\n```python\nfrom openai import OpenAI\n\nendpoint = \"https://api.gmi-serving.com/v1/\"\nmodel_name = \"XiaomiMiMo/MiMo-V2.6-Pro\"\n\napi_key = \"\u003cgmi-api-key\u003e\"\n\nclient = OpenAI(\n    base_url=f\"{endpoint}\",\n    api_key=api_key\n)\n\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_current_weather\",\n            \"description\": \"Get the current weather in a given location\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"location\": {\n                        \"type\": \"string\",\n                        \"description\": \"The city and state, e.g. San Francisco, CA\",\n                    },\n                    \"unit\": {\n                        \"type\": \"string\",\n                        \"enum\": [\"celsius\", \"fahrenheit\"]\n                    },\n                },\n                \"required\": [\"location\"],\n            },\n        }\n    }\n]\n\nmessages = [\n    {\n        \"role\": \"user\",\n        \"content\": \"What's the weather like in Boston today?\"\n    }\n]\n\ncompletion = client.chat.completions.create(\n    model=model_name,\n    messages=messages,\n    tools=tools,\n    tool_choice=\"auto\"\n)\n\nprint(completion)\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\n\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"XiaomiMiMo/MiMo-V2.6-Pro\",\n    \"messages\": [\n        {\n            \"role\": \"system\",\n            \"content\": \"You are a helpful AI assistant\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"List 3 countries and their capitals.\"\n        }\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\n\nprint(json.dumps(response.json(), indent=2))\n```\n````"},"model_price":{"modelName":"XiaomiMiMo/MiMo-V2.6-Pro","cacheHit":false,"pricePer1mPromptToken":326250,"pricePer1mCompletionToken":652500,"originPricePer1mCompletionToken":870000,"originPricePer1mPromptToken":435000,"percentageDiscount":2500,"effectiveAt":"2026-09-25T20:18:56.042808Z","endedAt":"","id":"f7809794-a853-40db-af8e-93c5d8b44053","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":326250,"outputPrice":652500,"cacheReadPrice":2700,"originInputPrice":435000,"originOutputPrice":870000,"originCacheReadPrice":3600}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"2a7f5f64-6c5f-4896-a524-8bcadaa3be7f","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-02-12T20:44:13.466474819Z","update_at":"2026-10-17T09:28:58.284440931Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMaxAI/MiniMax-M2.5","artifact_description":"MiniMaxAI/MiniMax-M2.5","artifact_tags":["LLM"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M2.5","artifact_template_id":"2a7f5f64-6c5f-4896-a524-8bcadaa3be7f","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":196608,"quantization":"fp8","info_config_map":[],"provider":"Minimax"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M2.5","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# MiniMax-M2.5\n\nMiniMax-M2.5 is an advanced reasoning model featuring built-in extended thinking capabilities for complex problem-solving. The model shows its reasoning process through \"thinking\" content blocks, making it ideal for tasks requiring step-by-step analysis, coding, and multi-step planning.\n\n## API Usage\n\nMiniMax-M2.5 supports two API formats:\n- **OpenAI-compatible**: /v1/chat/completions endpoint\n- **Anthropic-compatible**: /v1/messages endpoint (with extended thinking support)\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\", \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Anthropic-Compatible Endpoint (with Extended Thinking)\n\nThe /v1/messages endpoint provides access to the model's reasoning process through the \"thinking\" content block. You can also use the Anthropic SDK directly by configuring the base URL.\n\n#### Using Anthropic SDK (Recommended)\n\n```\n# Configure environment variables\nexport ANTHROPIC_BASE_URL=https://api.minimax.io/anthropic\nexport ANTHROPIC_API_KEY=${YOUR_API_KEY}\n```\n\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nmessage = client.messages.create(\n    model=\"MiniMax-M2.5\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"text\",\n                    \"text\": \"Hi, how are you?\"\n                }\n            ]\n        }\n    ]\n)\n\nfor block in message.content:\n    if block.type == \"thinking\":\n        print(f\"Thinking:\\n{block.thinking}\\n\")\n    elif block.type == \"text\":\n        print(f\"Text:\\n{block.text}\\n\")\n```\n\n#### Shell (Direct API)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Solve this step by step: What is 23 * 47?\"}\n    ]\n  }'\n```\n\n#### Shell (with Streaming)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Tell me a three sentence bedtime story about a unicorn.\"}\n    ],\n    \"stream\": true\n  }'\n```\n\n#### Python (with Streaming)\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nstream = client.messages.create(\n    model=\"MiniMax-M2.5\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Hi, how are you?\"}]}\n    ],\n    stream=True,\n)\n\nfor chunk in stream:\n    if chunk.type == \"content_block_delta\":\n        if hasattr(chunk, \"delta\") and chunk.delta:\n            if chunk.delta.type == \"thinking_delta\":\n                print(chunk.delta.thinking, end=\"\", flush=True)\n            elif chunk.delta.type == \"text_delta\":\n                print(chunk.delta.text, end=\"\", flush=True)\n```\n"},"model_price":{"modelName":"MiniMaxAI/MiniMax-M2.5","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":1200000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2026-07-17T19:05:16.050304Z","endedAt":"","id":"53fe2c95-a8d1-4587-b2d5-b4150d760995","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"claude","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":300000,"outputPrice":1200000,"cacheReadPrice":30000,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":30000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"8c94d723-b5bf-437a-8aec-a13358eb726d","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.8:glm-5-v7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-11T20:33:46.491179549Z","update_at":"2026-10-27T06:45:31.189220744Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMax AI: MiniMax-M2.5 (H200)","artifact_description":"MiniMaxAI/MiniMax-M2.5 B200","artifact_tags":["LLM","Text-to-Text","Text-Image-to-Text"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M2.5","artifact_template_id":"8c94d723-b5bf-437a-8aec-a13358eb726d","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":196608,"quantization":"fp8","info_config_map":[],"provider":"Minimax"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M2.5","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"top_k","value":0.9,"type":"number","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"number","min":0,"max":1,"step":0.1,"display_name":"Max Tokens","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"12","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"MiniMaxAI/MiniMax-M2.5","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"SERVE_COMMAND","value":"python -m sglang.launch_server --model-path MiniMaxAI/MiniMax-M2.5 --tp-size 4 --trust-remote-code --tool-call-parser minimax-m2 --reasoning-parser minimax-append-think --host 0.0.0.0 --port 8000 --mem-fraction-static 0.85 --grammar-backend xgrammar --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}' --max-running-requests 16 --max-queued-requests 12","is_hidden":true}]},"artifact_details":{"model_description":"# MiniMax-M2.5\n\nMiniMax-M2.5 is an advanced reasoning model featuring built-in extended thinking capabilities for complex problem-solving. The model shows its reasoning process through \"thinking\" content blocks, making it ideal for tasks requiring step-by-step analysis, coding, and multi-step planning.\n\n## API Usage\n\nMiniMax-M2.5 supports two API formats:\n- **OpenAI-compatible**: /v1/chat/completions endpoint\n- **Anthropic-compatible**: /v1/messages endpoint (with extended thinking support)\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\", \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Anthropic-Compatible Endpoint (with Extended Thinking)\n\nThe /v1/messages endpoint provides access to the model's reasoning process through the \"thinking\" content block. You can also use the Anthropic SDK directly by configuring the base URL.\n\n#### Using Anthropic SDK (Recommended)\n\n```\n# Configure environment variables\nexport ANTHROPIC_BASE_URL=https://api.minimax.io/anthropic\nexport ANTHROPIC_API_KEY=${YOUR_API_KEY}\n```\n\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nmessage = client.messages.create(\n    model=\"MiniMax-M2.5\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"text\",\n                    \"text\": \"Hi, how are you?\"\n                }\n            ]\n        }\n    ]\n)\n\nfor block in message.content:\n    if block.type == \"thinking\":\n        print(f\"Thinking:\\n{block.thinking}\\n\")\n    elif block.type == \"text\":\n        print(f\"Text:\\n{block.text}\\n\")\n```\n\n#### Shell (Direct API)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Solve this step by step: What is 23 * 47?\"}\n    ]\n  }'\n```\n\n#### Shell (with Streaming)\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: *************' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M2.5\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Tell me a three sentence bedtime story about a unicorn.\"}\n    ],\n    \"stream\": true\n  }'\n```\n\n#### Python (with Streaming)\n```\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nstream = client.messages.create(\n    model=\"MiniMax-M2.5\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Hi, how are you?\"}]}\n    ],\n    stream=True,\n)\n\nfor chunk in stream:\n    if chunk.type == \"content_block_delta\":\n        if hasattr(chunk, \"delta\") and chunk.delta:\n            if chunk.delta.type == \"thinking_delta\":\n                print(chunk.delta.thinking, end=\"\", flush=True)\n            elif chunk.delta.type == \"text_delta\":\n                print(chunk.delta.text, end=\"\", flush=True)\n```\n"},"model_price":{"modelName":"MiniMaxAI/MiniMax-M2.5","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":1200000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2026-07-17T19:05:16.050304Z","endedAt":"","id":"53fe2c95-a8d1-4587-b2d5-b4150d760995","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"claude","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":300000,"outputPrice":1200000,"cacheReadPrice":30000,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":30000}]},"endpoints":[]},{"artifact_id":"e779f6e7-b393-4208-9ec8-61ad2af3348c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-08T21:02:31.548438414Z","update_at":"2026-10-19T07:24:59.186684202Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"MiniMaxAI/MiniMax-M3.1-Flash","artifact_description":"MiniMaxAI/MiniMax-M3.1-Flash","artifact_tags":["LLM"],"artifact_volume_path":"models/MiniMaxAI/MiniMax-M3.1-Flash","artifact_template_id":"e779f6e7-b393-4208-9ec8-61ad2af3348c","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/minimax.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Minimax","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/MiniMaxAI/MiniMax-M3.1-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"\n# MiniMax-M3.1-Flash\n\nMiniMax-M3.1-Flash is a multimodal foundation model from MiniMax designed for high-speed reasoning, coding, tool use, and complex agentic workflows. It supports a 1M-token context window, enabling long-context processing and sustained multi-step tasks.\n\nBuilt on MiniMax Sparse Attention (MSA), the model uses efficient attention mechanisms to reduce computational overhead when processing long sequences. Its architecture is optimized for efficient inference, making it suitable for coding assistants, autonomous agents, multimodal understanding, and production-scale applications.\n\nMiniMax-M3.1-Flash is designed for interactive, multi-turn collaboration, with strong capabilities in software engineering, professional analysis, tool calling, and long-horizon task execution.\n\n## API Usage\n\nMiniMax-M3.1-Flash supports two API formats:\n- **OpenAI-compatible**: /v1/chat/completions endpoint\n- **Anthropic-compatible**: /v1/messages endpoint (with extended thinking support)\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer YOUR_API_KEY' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M3.1-Flash\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant with strong reasoning and coding ability.\"},\n      {\"role\": \"user\", \"content\": \"Explain how the Mixture-of-Experts architecture improves inference efficiency in large language models.\"}\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"MiniMaxAI/MiniMax-M3.1-Flash\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a capable AI coding assistant\"},\n        {\"role\": \"user\", \"content\": \"Refactor this multi-file Python module to make it async-ready\"}\n    ],\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Anthropic-Compatible Endpoint (with Extended Thinking)\n\nThe /v1/messages endpoint provides access to the model's reasoning process through the thinking content block. You can also use the Anthropic SDK directly by configuring the base URL.\n\n#### Using Anthropic SDK (Recommended)\n\n```bash\nexport ANTHROPIC_BASE_URL=https://api.gmi-serving.com\nexport ANTHROPIC_API_KEY=YOUR_API_KEY\n```\n\n```python\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nmessage = client.messages.create(\n    model=\"MiniMaxAI/MiniMax-M3.1-Flash\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"text\",\n                    \"text\": \"Hi, how are you?\"\n                }\n            ]\n        }\n    ]\n)\n\nfor block in message.content:\n    if block.type == \"thinking\":\n        print(f\"Thinking:\\n{block.thinking}\\n\")\n    elif block.type == \"text\":\n        print(f\"Text:\\n{block.text}\\n\")\n```\n\n#### Shell (Direct API)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: YOUR_API_KEY' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M3.1-Flash\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Solve this step by step: What is 23 * 47?\"}\n    ]\n  }'\n```\n\n#### Shell (with Streaming)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/messages \\\n  -H 'Content-Type: application/json' \\\n  -H 'x-api-key: YOUR_API_KEY' \\\n  --data '{\n    \"model\": \"MiniMaxAI/MiniMax-M3.1-Flash\",\n    \"max_tokens\": 1024,\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"Tell me a three sentence bedtime story about a unicorn.\"}\n    ],\n    \"stream\": true\n  }'\n```\n\n#### Python (with Streaming)\n\n```python\nimport anthropic\n\nclient = anthropic.Anthropic()\n\nstream = client.messages.create(\n    model=\"MiniMaxAI/MiniMax-M3.1-Flash\",\n    max_tokens=1000,\n    system=\"You are a helpful assistant.\",\n    messages=[\n        {\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": \"Hi, how are you?\"}]}\n    ],\n    stream=True,\n)\n\nfor chunk in stream:\n    if chunk.type == \"content_block_delta\":\n        if hasattr(chunk, \"delta\") and chunk.delta:\n            if chunk.delta.type == \"thinking_delta\":\n                print(chunk.delta.thinking, end=\"\", flush=True)\n            elif chunk.delta.type == \"text_delta\":\n                print(chunk.delta.text, end=\"\", flush=True)\n```\n  "},"model_price":{"modelName":"MiniMaxAI/MiniMax-M3.1-Flash","cacheHit":false,"pricePer1mPromptToken":112500,"pricePer1mCompletionToken":450000,"originPricePer1mCompletionToken":600000,"originPricePer1mPromptToken":150000,"percentageDiscount":2500,"effectiveAt":"2026-10-23T18:38:04.202513Z","endedAt":"","id":"17e01ba4-0b3f-40d7-9ac1-de1ef422b177","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":112500,"outputPrice":450000,"cacheReadPrice":11250,"originInputPrice":150000,"originOutputPrice":600000,"originCacheReadPrice":15000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"e6a2ea3d-ba41-49d0-af87-12b997c9cdb6","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-02-10T14:44:38.912092815Z","update_at":"2026-10-17T09:28:58.29948937Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi-K2.5","artifact_description":"moonshotai/Kimi-K2.5","artifact_tags":["LLM"],"artifact_volume_path":"models/moonshotai/Kimi-K2.5","artifact_template_id":"e6a2ea3d-ba41-49d0-af87-12b997c9cdb6","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Kimi"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/Kimi-K2.5","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Moonshotai Kimi-K2.5\n\nKimi K2.5 is an open-source, native multimodal agentic model built through continual pretraining on approximately 15 trillion mixed visual and text tokens atop Kimi-K2-Base. It seamlessly integrates vision and language understanding with advanced agentic capabilities, instant and thinking modes, as well as conversational and agentic paradigms.\n\n## API Usage\n\nYou can interact with the Moonshotai Kimi K2 Instruct 0905 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Moonshotai Kimi K2 Instruct 0905.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2.5\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"moonshotai/Kimi-K2.5\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"moonshotai/Kimi-K2.5","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":3000000,"originPricePer1mCompletionToken":3000000,"originPricePer1mPromptToken":600000,"percentageDiscount":0,"effectiveAt":"2026-02-07T07:59:59.98Z","endedAt":"","id":"074f0681-026f-4e1c-9071-a607dd4864b1","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"73286e78-971f-435b-876e-de0b3d5d8516","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-05-01T18:29:24.825223555Z","update_at":"2026-10-17T09:28:58.067334378Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi: K2.6","artifact_description":"moonshotai/Kimi: K2.6","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/moonshotai/Kimi-K2.6","artifact_template_id":"73286e78-971f-435b-876e-de0b3d5d8516","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/kimi.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Kimi"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/Kimi-K2.6","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Kimi-K2.6 \n\nKimi-K2.6 is an open-source, native multimodal agentic model developed by Moonshot AI. It is designed for long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration. The model supports both text and visual inputs and is optimized for advanced agentic and coding workflows.\n\n\n## API Usage\n\nYou can interact with the Kimi-K2.6 model through a RESTful API using an OpenAI-compatible interface. Kimi-K2.6 supports both Thinking mode and Instant mode.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the K2.6 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2.6\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful and capable AI coding assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2.6\",\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"moonshotai/Kimi-K2.6","cacheHit":false,"pricePer1mPromptToken":855000,"pricePer1mCompletionToken":3600000,"originPricePer1mCompletionToken":4000000,"originPricePer1mPromptToken":950000,"percentageDiscount":1000,"effectiveAt":"2026-04-20T20:20:06.036019Z","endedAt":"","id":"a0b30a09-7d1a-4cef-9161-fe5ba4f22d41","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":855000,"outputPrice":3600000,"cacheReadPrice":144000,"originInputPrice":950000,"originOutputPrice":4000000,"originCacheReadPrice":160000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"2b6c9f5e-7afa-4125-b512-63d2865dd903","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-06-12T20:05:58.551937405Z","update_at":"2026-10-17T09:28:57.99642718Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi: K2.7-Code","artifact_description":"moonshotai/Kimi: K2.7-Code","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/moonshotai/kimi-k2.7-code","artifact_template_id":"2b6c9f5e-7afa-4125-b512-63d2865dd903","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/kimi.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Kimi"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/kimi-k2.7-code","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Kimi-K2.7-Code \n\nKimi-K2.7-Code is an open-source, code-focused agentic model developed by Moonshot AI. It is designed for advanced software engineering workflows, including long-horizon coding tasks, code generation, debugging, refactoring, repository-level understanding, tool use, and agentic task execution.\n\nThe model supports text-based coding interactions and is optimized for developer productivity, autonomous coding workflows, and complex reasoning over codebases.\n\n\n## API Usage\n\nYou can interact with the Kimi-K2.7-Code model through a RESTful API using an OpenAI-compatible interface. Kimi-K2.7-Code is suitable for coding assistants, agentic development tools, debugging workflows, and automated code review systems.\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the Kimi-K2.7-Code model.\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k2.7-code\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful and capable AI coding assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k2.7-code\",\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"moonshotai/kimi-k2.7-code","cacheHit":false,"pricePer1mPromptToken":950000,"pricePer1mCompletionToken":4000000,"originPricePer1mCompletionToken":4000000,"originPricePer1mPromptToken":950000,"percentageDiscount":0,"effectiveAt":"2026-06-12T20:14:34.581104Z","endedAt":"","id":"fca1a0c7-9e76-43ae-abff-9505ec0a027e","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":950000,"outputPrice":4000000,"cacheReadPrice":190000,"originInputPrice":950000,"originOutputPrice":4000000,"originCacheReadPrice":190000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"c8f01e1e-b793-41f4-80e9-b6e9dc3f9da2","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-20T03:18:41.860710596Z","update_at":"2026-10-17T09:28:58.428805626Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi-K2-Instruct-0905","artifact_description":"moonshotai/Kimi-K2-Instruct-0905","artifact_tags":["LLM"],"artifact_volume_path":"models/moonshotai/Kimi-K2-Instruct-0905","artifact_template_id":"c8f01e1e-b793-41f4-80e9-b6e9dc3f9da2","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Kimi"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/Kimi-K2-Instruct-0905","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Moonshotai Kimi K2 Instruct 0905\n\nMoonshotai Kimi K2 Instruct 0905 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Moonshotai Kimi K2 Instruct 0905 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Moonshotai Kimi K2 Instruct 0905.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2-Instruct-0905\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"moonshotai/Kimi-K2-Instruct-0905\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"moonshotai/Kimi-K2-Instruct-0905","cacheHit":false,"pricePer1mPromptToken":570000,"pricePer1mCompletionToken":2290000,"originPricePer1mCompletionToken":2290000,"originPricePer1mPromptToken":570000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:09:55.585469Z","endedAt":"","id":"e9e59e12-d9e6-4c9f-9ec9-9686f722ff34","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":570000,"outputPrice":2290000,"cacheReadPrice":140000,"originInputPrice":570000,"originOutputPrice":2290000,"originCacheReadPrice":140000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"90e16d8b-6115-4e38-88cf-8fef0f5833e3","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-29T09:38:08.140743293Z","update_at":"2026-10-17T09:28:58.336640309Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/Kimi-K2-Thinking","artifact_description":"moonshotai/Kimi-K2-Thinking","artifact_tags":["LLM"],"artifact_volume_path":"models/moonshotai/Kimi-K2-Thinking","artifact_template_id":"90e16d8b-6115-4e38-88cf-8fef0f5833e3","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Kimi"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/Kimi-K2-Thinking","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Moonshotai Kimi K2 Instruct 0905\n\nMoonshotai Kimi K2 Instruct 0905 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Moonshotai Kimi K2 Instruct 0905 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Moonshotai Kimi K2 Instruct 0905.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/Kimi-K2-Instruct-0905\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"moonshotai/Kimi-K2-Instruct-0905\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"moonshotai/Kimi-K2-Thinking","cacheHit":false,"pricePer1mPromptToken":800000,"pricePer1mCompletionToken":1200000,"originPricePer1mCompletionToken":1200000,"originPricePer1mPromptToken":800000,"percentageDiscount":0,"effectiveAt":"2025-11-08T04:38:04.98Z","endedAt":"","id":"54ddc322-997f-43e6-a7cd-2c187b9e8e6b","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"370ac334-73cf-4625-b1a8-24761bfb1f70","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-26T04:21:37.427656166Z","update_at":"2026-10-26T05:00:56.109475126Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"moonshotai/kimi-k3.1-pro","artifact_description":"moonshotai/kimi-k3.1-pro","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/moonshotai/kimi-k3.1-pro","artifact_template_id":"370ac334-73cf-4625-b1a8-24761bfb1f70","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Kimi","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/moonshotai/kimi-k3.1-pro","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Kimi-K3.1-Pro\n\nKimi-K3.1-Pro is Moonshot AI's flagship open-weight multimodal reasoning model and the successor to Kimi-K3. Built on a large-scale Mixture-of-Experts architecture, it is designed for frontier-level performance across complex reasoning, long-horizon coding, and agentic workflows. The model features native image and video understanding, a 1,048,576-token context window, and enhanced capabilities for sustained, multi-step tasks.\n\nKimi-K3.1-Pro supports text, image, and video inputs with text outputs, including code and structured JSON. Thinking mode is always enabled, with configurable reasoning effort (low, high, and max), where max is the default. The model is optimized for autonomous software engineering, repository-scale coding, complex tool use, 3D creation, financial research, and end-to-end knowledge work. It also supports function calling and structured outputs, making it suitable for sophisticated AI agents and long-running development workflows.\n\n## API Usage\n\nYou can interact with the Kimi-K3.1-Pro model through a RESTful API using an OpenAI-compatible interface. Kimi-K3.1-Pro supports configurable reasoning effort through the reasoning_effort parameter (low, high, and max), with max used by default. The model uses fixed sampling settings of temperature: 1.0 and top_p: 0.95.\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the Kimi-K3.1-Pro model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k3.1-pro\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful and capable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"reasoning_effort\": \"high\",\n    \"max_completion_tokens\": 8000\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"moonshotai/kimi-k3.1-pro\",\n    \"reasoning_effort\": \"high\",\n    \"max_completion_tokens\": 8000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```\n\n## Model Capabilities\n\n- **Context Window:** 1,048,576 tokens, shared between input and output.\n- **Maximum Output:** 131,072 tokens by default, configurable up to 1,048,576 tokens within the context limit.\n- **Input Modalities:** Text, image, and video.\n- **Output Modalities:** Text, code, and structured JSON.\n- **Reasoning:** Always-on thinking with `reasoning_effort` values of `low`, `high`, and `max`.\n- **Tool Calling:** Supports function calling and multi-step agent workflows.\n- **Structured Outputs:** Supports JSON mode and JSON Schema.\n- **Streaming:** Supports incremental generation of reasoning and final responses.\n\n**Note:** Reasoning tokens count toward the output token limit. Applications should allocate sufficient output tokens for both reasoning and final answers."},"model_price":{"modelName":"moonshotai/kimi-k3.1-pro","cacheHit":false,"pricePer1mPromptToken":2000000,"pricePer1mCompletionToken":10000000,"originPricePer1mCompletionToken":10000000,"originPricePer1mPromptToken":2000000,"percentageDiscount":0,"effectiveAt":"2026-10-29T06:54:32.821076Z","endedAt":"","id":"fe67d44a-4f50-4e30-98a1-683735764cef","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"claudeV2","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":2000000,"outputPrice":10000000,"cacheReadPrice":200000,"ephemeral5mInputPrice":2000000,"ephemeral1hInputPrice":4000000,"originInputPrice":2000000,"originOutputPrice":10000000,"originCacheReadPrice":200000,"originEphemeral5mInputPrice":2000000,"originEphemeral1hInputPrice":4000000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"d742e342-4051-4098-9401-b0de97a6ecbb","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_nemotron3_nano_omni:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-06-02T01:24:50.091816389Z","update_at":"2026-10-17T09:28:58.009319819Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Nemotron 3 ultra","artifact_description":"Nemotron 3 ultra","artifact_tags":["LLM","Text-Image-to-Text"],"artifact_volume_path":"models/nvidia/nemotron-3-ultra-550b-a55b","artifact_template_id":"d742e342-4051-4098-9401-b0de97a6ecbb","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/Nvidia.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Nvidia","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/nvidia/nemotron-3-ultra-550b-a55b","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Nemotron 3 Ultra\n\nNemotron 3 Ultra is a frontier-scale large language model developed by NVIDIA, designed for high-performance reasoning, agentic workflows, long-context analysis, tool use, and advanced conversational AI. Built on NVIDIA’s hybrid Latent Mixture-of-Experts architecture, it features 550B total parameters with 55B active parameters, enabling strong efficiency while delivering frontier-level capabilities across complex reasoning, code, math, science, multilingual tasks, and high-stakes analytical workloads.\n\nThe model is well-suited for demanding AI applications such as intelligent assistants, coding agents, enterprise RAG systems, multi-step agent workflows, long-document analysis, structured reasoning, and advanced tool-calling scenarios.\n\n\n## API Usage\n\nYou can interact with the Nemotron 3 Ultra model through a RESTful OpenAI-compatible API. The examples below demonstrate how to call the model for text generation, reasoning control, and function calling.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the Nemotron 3 Ultra model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"nvidia/nemotron-3-ultra-550b-a55b\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.2,\n    \"max_tokens\": 800\n  }'\n```\n\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"nvidia/nemotron-3-ultra-550b-a55b\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\"symbol\"]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```"},"model_price":{"modelName":"nvidia/nemotron-3-ultra-550b-a55b","cacheHit":false,"pricePer1mPromptToken":800000,"pricePer1mCompletionToken":2600000,"originPricePer1mCompletionToken":2600000,"originPricePer1mPromptToken":800000,"percentageDiscount":0,"effectiveAt":"2026-06-05T00:52:42.379587Z","endedAt":"","id":"46dda23b-23bd-4b21-9df6-28a83e4cc3b1","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":800000,"outputPrice":2600000,"cacheReadPrice":100000,"originInputPrice":800000,"originOutputPrice":2600000,"originCacheReadPrice":100000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"b67c62b3-6652-4412-8dcb-be37125b7bbb","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.6.post2:v1.5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-05-02T23:10:54.537050905Z","update_at":"2026-10-27T06:45:31.824786866Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"OpenAI: CLIP Vit Large Patch14","artifact_description":"","artifact_tags":["Embedding","CLIP"],"artifact_volume_path":"models/laion/CLIP-ViT-B-32-laion2B-s34B-b79K","artifact_template_id":"b67c62b3-6652-4412-8dcb-be37125b7bbb","artifact_icon_link":"https://cdn-avatars.huggingface.co/v1/production/uploads/1665438593823-617c01b3274d1c00141c2635.png","preview_image_url":"","is_public":true,"model_info":{"context_length":0,"quantization":"fp16","info_config_map":null,"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/laion/CLIP-ViT-B-32-laion2B-s34B-b79K","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[],"env_parameters":[{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"100","is_hidden":false},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path laion/CLIP-ViT-B-32-laion2B-s34B-b79K","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false}]},"artifact_details":{"model_description":"# CLIP-ViT-B-32-laion2B-s34B-b79K\n\nA CLIP ViT-B/32 model trained with the LAION-2B English subset of LAION-5B (https://laion.ai/blog/laion-5b/) using OpenCLIP (https://github.com/mlfoundations/open_clip).\n\n## API Examples\n\n### Shell\n```\ncurl --location 'https://api.gmi-serving.com/v1/embeddings' \\\n--header 'Content-Type: application/json' \\\n--header 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36' \\\n-H 'Authorization: Bearer *************' \\\n--data '{\n      \"model\": \"laion/CLIP-ViT-B-32-laion2B-s34B-b79K\",\n      \"input\": \"A photo of a cat sitting on a table\"\n}'\n```\n"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"ed1f3df9-83a6-4ca5-a974-9d113017409c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-21T09:39:12.629093173Z","update_at":"2026-10-17T09:28:58.387553338Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Proxy: zai-org/GLM-4.7","artifact_description":"zai-org/GLM-4.7-FP8","artifact_tags":["LLM"],"artifact_volume_path":"models/zai-org/GLM-4.7-FP8","artifact_template_id":"ed1f3df9-83a6-4ca5-a974-9d113017409c","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"fp8","info_config_map":[],"provider":"Zhipu"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-4.7-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# DeepSeek-V3.2\n\nDeepSeek-V3.2 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the DeepSeek R1 0528 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of DeepSeek-V3.2.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"deepseek-ai/DeepSeek-V3.2\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"deepseek-ai/DeepSeek-V3.2\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"zai-org/GLM-4.7-FP8","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":2200000,"originPricePer1mCompletionToken":2200000,"originPricePer1mPromptToken":600000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:13:29.58179Z","endedAt":"","id":"62a22cdc-8210-42f8-a4b7-586fccee1f17","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":600000,"outputPrice":2200000,"cacheReadPrice":110000,"originInputPrice":600000,"originOutputPrice":2200000,"originCacheReadPrice":110000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"a013cb9e-9e7c-4ab2-9b15-d4a86fe4827c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.14_2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-04-28T21:51:29.352958451Z","update_at":"2026-10-27T06:45:31.844393981Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 235B A22B FP8","artifact_description":"Qwen/Qwen3-235B-A22B-FP8","artifact_tags":["LLM","Text-to-Text","Qwen"],"artifact_volume_path":"models/Qwen/Qwen3-235B-A22B-FP8","artifact_template_id":"a013cb9e-9e7c-4ab2-9b15-d4a86fe4827c","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":32768,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-235B-A22B-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"20","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"gpu_scheduler","value":"true","is_hidden":false},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-235B-A22B-FP8  --tp-size 2 --reasoning-parser qwen3 --mem-fraction-static 0.85 --trust-remote-code --tool-call-parser qwen25 --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# Qwen3-235B-A22B-FP8\n\nQwen3-235B-A22B-FP8 is a powerful large language model while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3-235B-A22B-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-235B-A22B-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-235B-A22B-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n```\n# example for function call \ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-235B-A22B-FP8\",\n    \"messages\": [\n        {\n            \"role\": \"system\",\n            \"content\": \"You are a helpful assistant.\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather like in San Francisco?\"\n        }\n    ],\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get the current weather in a given location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city and state, e.g., San Francisco, CA\"\n                        },\n                        \"unit\": {\n                            \"type\": \"string\",\n                            \"enum\": [\n                                \"celsius\",\n                                \"fahrenheit\"\n                            ],\n                            \"description\": \"The temperature unit to use\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"tool_choice\": \"auto\"\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-235B-A22B-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-235B-A22B-FP8","cacheHit":false,"pricePer1mPromptToken":170000,"pricePer1mCompletionToken":1090000,"originPricePer1mCompletionToken":1090000,"originPricePer1mPromptToken":170000,"percentageDiscount":0,"effectiveAt":"2025-04-28T03:17:51.689126Z","endedAt":"","id":"7c5a6df4-7c06-41a9-9d5b-02a72da9f476","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"b464533f-fe7c-4c2b-849b-8d913169584f","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-07-22T04:33:07.114762802Z","update_at":"2026-10-27T06:45:31.796281454Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 235B A22B Instruct 2507 FP8","artifact_description":"Qwen: Qwen3 235B A22B Instruct 2507 FP8","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","artifact_template_id":"b464533f-fe7c-4c2b-849b-8d913169584f","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"20","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"20","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.4.7:v1.5.8","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 --tp-size 4 --mem-fraction-static 0.9 --trust-remote-code --tool-call-parser qwen25 --port 8000 --allow-auto-truncate","is_hidden":true}]},"artifact_details":{"model_description":"# Qwen3-235B-A22B-Instruct-2507-FP8\n\nQwen3-235B-A22B-Instruct-2507-FP8 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3-235B-A22B-Instruct-2507-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-235B-A22B-Instruct-2507-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","cacheHit":false,"pricePer1mPromptToken":87500,"pricePer1mCompletionToken":350000,"originPricePer1mCompletionToken":1400000,"originPricePer1mPromptToken":350000,"percentageDiscount":7500,"effectiveAt":"2026-04-10T06:07:41.903599Z","endedAt":"","id":"db82e275-a21f-4730-b84d-99333d8e9e0b","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":87500,"outputPrice":350000,"cacheReadPrice":17500,"originInputPrice":350000,"originOutputPrice":1400000,"originCacheReadPrice":70000}]},"endpoints":[]},{"artifact_id":"8beab62a-79de-4d64-adf9-4abe6d96e7f3","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-07-25T18:28:35.523211414Z","update_at":"2026-10-27T06:45:31.788913356Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 235B A22B Thinking 2507 FP8","artifact_description":"Qwen: Qwen3 235B A22B Thinking 2507 FP8","artifact_tags":["LLM","Text-to-Text","Qwen3 Thinking"],"artifact_volume_path":"models/Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","artifact_template_id":"8beab62a-79de-4d64-adf9-4abe6d96e7f3","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":131072,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"5","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.0:v1.3.2","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-235B-A22B-Thinking-2507-FP8 --tp-size 4 --mem-fraction-static 0.85 --trust-remote-code --tool-call-parser qwen25 --reasoning-parser qwen3-thinking --port 8000 --chat-template ./chat_template/qwen3_235B_A22B_thinking_2507.jinja","is_hidden":true}]},"artifact_details":{"model_description":""},"model_price":{"modelName":"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","cacheHit":false,"pricePer1mPromptToken":350000,"pricePer1mCompletionToken":4200000,"originPricePer1mCompletionToken":4200000,"originPricePer1mPromptToken":350000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:08:47.039237Z","endedAt":"","id":"9bedb5b3-5ca3-4d1f-a5fa-a719aa276e8f","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"e8c203de-a4af-4026-a5db-cbbfc0b82fa7","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.14_2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-09-10T21:10:27.631048597Z","update_at":"2026-10-27T06:45:31.65804401Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3-30B-A3B","artifact_description":"Qwen: Qwen3-30B-A3B","artifact_tags":["LLM","Qwen","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-30B-A3B","artifact_template_id":"e8c203de-a4af-4026-a5db-cbbfc0b82fa7","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":40946,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-30B-A3B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.7,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":32768,"type":"numeric","min":1,"max":38912,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":20,"type":"numeric","min":0,"max":20,"step":1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":1,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"RATE_LIMIT","value":"50000","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"5","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-30B-A3B","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-30B-A3B --reasoning-parser qwen3 --mem-fraction-static 0.9 --trust-remote-code --tool-call-parser qwen25 --enable-metrics --port 8000 --chat-template ./chat_template/qwen3_30B_A3B.jinja","is_hidden":true}]},"artifact_details":{"model_description":"# QwQ-32B\n\nQwQ-32B is a powerful large language model while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the QwQ-32B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of QwQ-32B.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/QwQ-32B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/QwQ-32B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-30B-A3B","cacheHit":false,"pricePer1mPromptToken":80000,"pricePer1mCompletionToken":250000,"originPricePer1mCompletionToken":250000,"originPricePer1mPromptToken":80000,"percentageDiscount":0,"effectiveAt":"2025-09-10T08:38:04.98Z","endedAt":"","id":"52ddf1cf-dfbf-4481-92b7-e6f05be27d81","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"ae9cad4b-9b87-4b80-99e2-4a73f5715a87","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.14_2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-04-28T22:03:12.674358421Z","update_at":"2026-10-27T06:45:31.837780553Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 32B FP8","artifact_description":"Qwen/Qwen3-32B-FP8","artifact_tags":["LLM","Text-to-Text","Qwen"],"artifact_volume_path":"models/Qwen/Qwen3-32B-FP8","artifact_template_id":"ae9cad4b-9b87-4b80-99e2-4a73f5715a87","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":32768,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-32B-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":1,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":0,"step":0,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"20","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"30","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"10","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-32B-FP8","is_hidden":false},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":false},{"key":"gpu_scheduler","value":"true","is_hidden":false},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-32B-FP8 --reasoning-parser qwen3 --mem-fraction-static 0.8 --trust-remote-code --tool-call-parser qwen25 --enable-metrics --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# Qwen3-32B-FP8\n\nQwen3-32B-FP8 is a powerful large language model while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3-32B-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-32B-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-32B-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n```\n# example for function call \ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-235B-A22B-FP8\",\n    \"messages\": [\n        {\n            \"role\": \"system\",\n            \"content\": \"You are a helpful assistant.\"\n        },\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the weather like in San Francisco?\"\n        }\n    ],\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get the current weather in a given location\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city and state, e.g., San Francisco, CA\"\n                        },\n                        \"unit\": {\n                            \"type\": \"string\",\n                            \"enum\": [\n                                \"celsius\",\n                                \"fahrenheit\"\n                            ],\n                            \"description\": \"The temperature unit to use\"\n                        }\n                    },\n                    \"required\": [\n                        \"location\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"tool_choice\": \"auto\"\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-32B-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-32B-FP8","cacheHit":false,"pricePer1mPromptToken":100000,"pricePer1mCompletionToken":600000,"originPricePer1mCompletionToken":600000,"originPricePer1mPromptToken":100000,"percentageDiscount":0,"effectiveAt":"2025-04-28T03:17:55.41417Z","endedAt":"","id":"ce3d042a-b219-4336-905d-f697b4bb4906","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"b0c97d0a-6272-499b-a12c-e860cbb66d6d","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-05T05:22:13.341207403Z","update_at":"2026-10-17T09:28:58.233023452Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.5 122B A10B","artifact_description":"Qwen3.5-122B-A10B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.5-122B-A10B","artifact_template_id":"b0c97d0a-6272-499b-a12c-e860cbb66d6d","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.5-122B-A10B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.5 122B A10B\n\nQwen3.5 122B A10B is a native vision-language model built on a hybrid architecture that integrates a linear attention mechanism with a sparse mixture-of-experts model, achieving higher inference efficiency. It excels in complex Agent tasks such as multi-step reasoning, tool calling, and cross-application operations, closing the gap between mid-size open-source models and top-tier closed-source models.\n\n## API Usage\n\nYou can interact with the Qwen3.5 122B A10B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.5 122B A10B.\n\n### Shell\n```\ncurl --request POST \\\n--url https://api.gmi-serving.com/v1/chat/completions \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: Bearer *************' \\\n  --data '{\n  \"model\": \"Qwen/Qwen3.5-122B-A10B\",\n  \"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n  {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n  ],\n  \"temperature\": 0,\n  \"max_completion_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.5-122B-A10B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.5-122B-A10B","cacheHit":false,"pricePer1mPromptToken":400000,"pricePer1mCompletionToken":3200000,"originPricePer1mCompletionToken":3200000,"originPricePer1mPromptToken":400000,"percentageDiscount":0,"effectiveAt":"2026-03-05T09:00:04.98Z","endedAt":"","id":"51e4e652-e0e2-4c1b-b0ea-24423a47d96e","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"b5ca6de9-fc59-4bb7-a0b6-646e4ca2bc72","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-05T02:14:44.113437149Z","update_at":"2026-10-17T09:28:58.239971032Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.5 27B","artifact_description":"Qwen3.5-27B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.5-27B","artifact_template_id":"b5ca6de9-fc59-4bb7-a0b6-646e4ca2bc72","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.5-27B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.5 27B\n\nQwen3.5 27B is a powerful dense language model that incorporates a linear attention mechanism, delivering fast response times while balancing inference speed and performance. \n\n## API Usage\n\nYou can interact with the Qwen3.5 27B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.5 27B.\n\n### Shell\n```\ncurl --request POST \\\n--url https://api.gmi-serving.com/v1/chat/completions \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: Bearer *************' \\\n  --data '{\n  \"model\": \"Qwen/Qwen3.5-27B\",\n  \"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n  {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n  ],\n  \"temperature\": 0,\n  \"max_completion_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.5-27B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.5-27B","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":2400000,"originPricePer1mCompletionToken":2400000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2026-03-05T09:00:04.98Z","endedAt":"","id":"e3dc4158-8d89-4896-8cd9-5d75fec7474a","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"dda5db69-e0a0-444d-88b5-862d79c85a0f","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-05T05:24:50.859798552Z","update_at":"2026-10-17T09:28:58.226476912Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.5 35B A3B","artifact_description":"Qwen/Qwen3.5-35B-A3B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.5-35B-A3B","artifact_template_id":"dda5db69-e0a0-444d-88b5-862d79c85a0f","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.5-35B-A3B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.5 35B A3B\n\nQwen3.5-35B-A3B is a native vision-language model designed with a hybrid architecture that integrates linear attention mechanisms and a sparse mixture-of-experts model. It surpasses both Qwen3-235B-A22B and Qwen3-VL-235B-A22B, demonstrating that the key to intelligence breakthroughs lies not in scaling parameters, but in the effective synergy of better architecture, higher-quality data, and reinforcement learning.\n## API Usage\n\nYou can interact with the Qwen3.5 35B A3B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.5 35B A3B.\n\n### Shell\n```\ncurl --request POST \\\n--url https://api.gmi-serving.com/v1/chat/completions \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: Bearer *************' \\\n  --data '{\n  \"model\": \"Qwen/Qwen3.5-35B-A3B\",\n  \"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n  {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n  ],\n  \"temperature\": 0,\n  \"max_completion_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.5-35B-A3B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.5-35B-A3B","cacheHit":false,"pricePer1mPromptToken":250000,"pricePer1mCompletionToken":2000000,"originPricePer1mCompletionToken":2000000,"originPricePer1mPromptToken":250000,"percentageDiscount":0,"effectiveAt":"2026-03-05T09:00:04.98Z","endedAt":"","id":"55845dfc-bb89-41c3-a441-c0674ab91f3c","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"29ac1bb2-9c03-4a7c-9178-fb438d0e1814","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-05T05:28:36.241679099Z","update_at":"2026-10-17T09:28:58.218917623Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.5 397B A17B","artifact_description":"Qwen/Qwen3.5-397B-A17B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.5-397B-A17B","artifact_template_id":"29ac1bb2-9c03-4a7c-9178-fb438d0e1814","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[{"key":"output_length","value":65536,"type":"numeric","min":0,"max":0,"step":0,"display_name":"Output length","category":"general"}],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.5-397B-A17B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.5 397B A17B\n\nQwen3.5-397B-A17B is the flagship model of the Qwen3.5 series, based on a hybrid architecture that integrates linear attention mechanisms with sparse Mixture-of-Experts (MoE), achieving higher inference efficiency. It demonstrates exceptional performance across a wide range of tasks including language understanding, logical reasoning, code generation, agentic tasks, image understanding, video understanding, and GUI interaction, comparable to current top-tier frontier models. With robust code generation and agentic capabilities, it exhibits strong generalization across various agent scenarios.\n\n## API Usage\n\nYou can interact with the Qwen3.5 397B A17B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.5 397B A17B.\n\n### Shell\n```\ncurl --request POST \\\n--url https://api.gmi-serving.com/v1/chat/completions \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: Bearer *************' \\\n  --data '{\n  \"model\": \"Qwen/Qwen3.5-397B-A17B\",\n  \"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n  {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n  ],\n  \"temperature\": 0,\n  \"max_completion_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.5-397B-A17B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.5-397B-A17B","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":3600000,"originPricePer1mCompletionToken":3600000,"originPricePer1mPromptToken":600000,"percentageDiscount":0,"effectiveAt":"2026-03-05T09:00:04.98Z","endedAt":"","id":"1d056165-eb79-44bc-b4f8-f687b9ccd30c","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"9deb2533-b297-4af8-825f-cbdd67130ee2","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-05-20T20:15:32.625982015Z","update_at":"2026-10-17T09:28:58.047966208Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.6-35B-A3B","artifact_description":"Qwen/Qwen3.6-35B-A3B","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.6-35B-A3B","artifact_template_id":"9deb2533-b297-4af8-825f-cbdd67130ee2","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.6-35B-A3B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.6-35B-A3B\n\nQwen3.6-35B-A3B is an open-weight Mixture-of-Experts (MoE) model in the Qwen3.6 series. It features 35B total parameters with around 3B activated parameters, offering strong efficiency while maintaining advanced reasoning, coding, agentic workflow, and multimodal capabilities. The model uses a hybrid architecture with Gated DeltaNet linear attention, gated attention layers, and MoE blocks, and supports long-context use cases with a native 262K context window.\n\n## API Usage\n\nYou can interact with the Qwen3.6-35B-A3B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.6-35B-A3B.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3.6-35B-A3B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.6-35B-A3B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant.\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.6-35B-A3B","cacheHit":false,"pricePer1mPromptToken":248000,"pricePer1mCompletionToken":1485000,"originPricePer1mCompletionToken":1485000,"originPricePer1mPromptToken":248000,"percentageDiscount":0,"effectiveAt":"2026-05-20T20:11:59.778627Z","endedAt":"","id":"d05694e5-b947-4223-b88e-325fc3263821","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"c6ce4ef6-245b-4226-99e0-b6ead0530368","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-14T20:17:01.691911591Z","update_at":"2026-10-17T09:28:58.133135815Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.6 Plus","artifact_description":"Qwen/Qwen3.6-Plus","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.6-Plus","artifact_template_id":"c6ce4ef6-245b-4226-99e0-b6ead0530368","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.6-Plus","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.6 Plus\n\nQwen3.6 Plus is an advanced model in the Qwen3.6 series, designed with enhanced efficiency and performance through improvements in hybrid attention mechanisms and optimized Mixture-of-Experts (MoE) architecture. It delivers strong capabilities across a wide range of tasks including language understanding, logical reasoning, code generation, agentic workflows, image understanding, video understanding, and GUI interaction, matching or exceeding current frontier-level models. With improved robustness in coding and agent-based scenarios, it demonstrates excellent generalization and reliability across diverse real-world applications.\n\n## API Usage\n\nYou can interact with the Qwen3.6 Plus model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.6 Plus.\n\n### Shell\n```\ncurl --request POST \\\n--url https://api.gmi-serving.com/v1/chat/completions \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: Bearer *************' \\\n  --data '{\n  \"model\": \"Qwen/Qwen3.6-Plus\",\n  \"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n  {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n  ],\n  \"temperature\": 0,\n  \"max_completion_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.6-Plus\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.6-Plus","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":3000000,"originPricePer1mCompletionToken":3000000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2026-04-14T19:15:48.845991Z","endedAt":"","id":"3a21e467-520e-4e05-b6d7-8e66f5012266","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"8d9958e3-5f8d-48b2-a918-f077385757f6","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-14T20:22:39.678879807Z","update_at":"2026-10-17T09:28:58.127011776Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.6 Plus 2026-04-02","artifact_description":"Qwen/Qwen3.6-Plus-2026-04-02","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.6-Plus-2026-04-02","artifact_template_id":"8d9958e3-5f8d-48b2-a918-f077385757f6","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.6-Plus-2026-04-02","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.6 Plus\n\nQwen3.6 Plus is an advanced model in the Qwen3.6 series, designed with enhanced efficiency and performance through improvements in hybrid attention mechanisms and optimized Mixture-of-Experts (MoE) architecture. It delivers strong capabilities across a wide range of tasks including language understanding, logical reasoning, code generation, agentic workflows, image understanding, video understanding, and GUI interaction, matching or exceeding current frontier-level models. With improved robustness in coding and agent-based scenarios, it demonstrates excellent generalization and reliability across diverse real-world applications.\n\n## API Usage\n\nYou can interact with the Qwen3.6 Plus model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.6 Plus.\n\n### Shell\n```\ncurl --request POST \\\n--url https://api.gmi-serving.com/v1/chat/completions \\\n-H 'Content-Type: application/json' \\\n-H 'Authorization: Bearer *************' \\\n  --data '{\n  \"model\": \"Qwen/Qwen3.6-Plus\",\n  \"messages\": [\n  {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n  {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n  ],\n  \"temperature\": 0,\n  \"max_completion_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.6-Plus\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3.6-Plus-2026-04-02","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":3000000,"originPricePer1mCompletionToken":3000000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2026-04-14T19:16:16.101439Z","endedAt":"","id":"22056827-da61-4899-9ba6-d47a28e43c03","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"e7cd2464-04a4-4b85-bfbb-8caedcdca20c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-26T19:44:06.426612138Z","update_at":"2026-10-17T09:28:57.776732077Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.8 Flash","artifact_description":"Qwen/Qwen3.8-Flash","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.8-Flash","artifact_template_id":"e7cd2464-04a4-4b85-bfbb-8caedcdca20c","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048575,"quantization":"bf16","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.8-Flash","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.8 27B\n\nQwen3.8 27B is the compact, deployment-friendly dense model in the Qwen3.8 series — a 27-billion-parameter native vision-language model built for the agent era. It builds upon Qwen3.6-27B with key improvements in coding and office productivity across text and visual modalities, enabling more reliable end-to-end completion of complex tasks. It supports native multimodal input (text, image, video), a 1 million token context window, and stronger long-horizon autonomous execution for coding, professional workflows, and research. It supports hybrid thinking modes (enabled by default), tool calling, MCP integrations, explicit caching, and OpenAI/Anthropic-compatible APIs across agent frameworks (e.g., Claude Code, Qwen Code, OpenClaw). Text-only output.\n\n## API Usage\n\nYou can interact with the Qwen3.8 27B model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.8 27B.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3.8-27B\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.8-27B\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"Qwen/Qwen3.8-Flash","cacheHit":false,"pricePer1mPromptToken":160000,"pricePer1mCompletionToken":470000,"originPricePer1mCompletionToken":470000,"originPricePer1mPromptToken":160000,"percentageDiscount":0,"effectiveAt":"2026-08-26T19:52:49.737385Z","endedAt":"","id":"f6ecdc33-14c3-435e-b341-99e1be06fd97","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":160000,"outputPrice":470000,"cacheReadPrice":16000,"cacheWritePrice":200000,"originInputPrice":160000,"originOutputPrice":470000,"originCacheReadPrice":16000,"originCacheWritePrice":200000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"b2d7b50b-eafe-4174-bbbf-9ea95f248bf8","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-09-08T22:39:16.399430938Z","update_at":"2026-10-17T09:28:57.716846668Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3.8 Max 0902","artifact_description":"Qwen/Qwen3.8-Max-0902","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3.8-Max-0902","artifact_template_id":"b2d7b50b-eafe-4174-bbbf-9ea95f248bf8","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3.8-Max-0902","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3.8 Max\n\nQwen3.8 Max 0902 is the largest and most capable model in the Qwen3.8 series — a 2.4-trillion-parameter sparse MoE flagship (95B active per query) built for the agent era. It extends Qwen3.7 Max with native multimodal input (text, image, video), a 1 million token context window, and stronger long-horizon autonomous execution for coding, professional workflows, and research. It supports hybrid thinking modes (enabled by default), tool calling, MCP integrations, explicit caching, and OpenAI/Anthropic-compatible APIs across agent frameworks (e.g., Claude Code, Qwen Code, OpenClaw). Text-only output.\n\n## API Usage\n\nYou can interact with the Qwen3.8 Max-0902 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3.8 Max-0902.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3.8-Max-0902\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n  }'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3.8-Max-0902\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_completion_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"Qwen/Qwen3.8-Max-0902","cacheHit":false,"pricePer1mPromptToken":1650000,"pricePer1mCompletionToken":4951000,"originPricePer1mCompletionToken":4951000,"originPricePer1mPromptToken":1650000,"percentageDiscount":0,"effectiveAt":"2026-09-10T20:16:06.493653Z","endedAt":"","id":"af59c9b1-d11b-4470-bdfe-e1aa025d7bfd","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1650000,"outputPrice":4951000,"cacheReadPrice":206000,"cacheWritePrice":2063000,"originInputPrice":1650000,"originOutputPrice":4951000,"originCacheReadPrice":206000,"originCacheWritePrice":2063000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"13ee0563-2204-420d-8178-660b25b61e66","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-10-09T09:44:47.038441178Z","update_at":"2026-10-27T06:45:31.54716819Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3-Coder-30B-A3B-Instruct-FP8","artifact_description":"Qwen: Qwen3-Coder-30B-A3B-Instruct-FP8","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8","artifact_template_id":"13ee0563-2204-420d-8178-660b25b61e66","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"20","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"4","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 --trust-remote-code --tool-call-parser qwen3_coder --port 8000 --allow-auto-truncate","is_hidden":true}]},"artifact_details":{"model_description":"# Qwen3-Coder-30B-A3B-Instruct-FP8\n\nQwen3-Coder is available in multiple sizes. Today, we're excited to introduce Qwen3-Coder-30B-A3B-Instruct-FP8. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements:\n\nSignificant Performance among open models on Agentic Coding, Agentic Browser-Use, and other foundational coding tasks.\nLong-context Capabilities with native support for 256K tokens, extendable up to 1M tokens using Yarn, optimized for repository-scale understanding.\nAgentic Coding supporting for most platform such as Qwen Code, CLINE, featuring a specially designed function call format.\n## API Usage\n\nYou can interact with the Qwen3-Coder-30B-A3B-Instruct-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-Coder-30B-A3B-Instruct-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"b8093904-fab6-457b-86fb-22c65e59c911","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-20T21:00:57.748176523Z","update_at":"2026-10-27T06:45:31.384016193Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 Coder 480B A35B Instruct FP8","artifact_description":"Qwen: Qwen3 Coder 480B A35B Instruct FP8","artifact_tags":["Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","artifact_template_id":"b8093904-fab6-457b-86fb-22c65e59c911","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max Tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""}],"env_parameters":[{"key":"MAAS_OR_DEDICATED","value":"maas","is_hidden":false},{"key":"TARGET_ONGOING_REQUESTS","value":"16","is_hidden":false},{"key":"ENV","value":"prod","is_hidden":false},{"key":"MAX_ONGOING_REQUESTS","value":"24","is_hidden":false},{"key":"MAX_QUEUED_REQUESTS","value":"16","is_hidden":false},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":false},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":false},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.4:v1.5.ray_fix_v1_otel_qwen3_coder_480b","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 --tp-size 8 --ep-size 8 --mem-fraction-static 0.9 --trust-remote-code --tool-call-parser qwen3_coder --chat-template ./chat_template/qwen3_coder.jinja --port 8000 --grammar-backend xgrammar --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}' --max-running-requests 24 --max-queued-requests 24","is_hidden":true}]},"artifact_details":{"model_description":""},"model_price":{"modelName":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","cacheHit":false,"pricePer1mPromptToken":900000,"pricePer1mCompletionToken":4500000,"originPricePer1mCompletionToken":4500000,"originPricePer1mPromptToken":900000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:28:05.196427Z","endedAt":"","id":"b841d686-b6ab-4461-8d31-63366a3a2cc2","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":900000,"outputPrice":4500000,"cacheReadPrice":180000,"originInputPrice":900000,"originOutputPrice":4500000,"originCacheReadPrice":180000},{"threshold":32000,"inputPrice":1620000,"outputPrice":8100000,"cacheReadPrice":324000,"originInputPrice":1620000,"originOutputPrice":8100000,"originCacheReadPrice":324000},{"threshold":128000,"inputPrice":2700000,"outputPrice":13500000,"cacheReadPrice":540000,"originInputPrice":2700000,"originOutputPrice":13500000,"originCacheReadPrice":540000}]},"endpoints":[]},{"artifact_id":"73a40add-f8bc-4bec-8097-8b24cde9b8aa","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-03T01:15:19.080089572Z","update_at":"2026-10-17T09:28:58.254361992Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 Next 80B A3B Instruct ","artifact_description":"Qwen: Qwen3 Next 80B A3B Instruct","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-Next-80B-A3B-Instruct","artifact_template_id":"73a40add-f8bc-4bec-8097-8b24cde9b8aa","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-Next-80B-A3B-Instruct","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3 Next 80B A3B Instruct\n\nQwen3 Next 80B A3B Instruct is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3 Next 80B A3B Instruct model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3 Next 80B A3B Instruct.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-Next-80B-A3B-Instruct\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-Next-80B-A3B-Instruct\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-Next-80B-A3B-Instruct","cacheHit":false,"pricePer1mPromptToken":150000,"pricePer1mCompletionToken":1500000,"originPricePer1mCompletionToken":1500000,"originPricePer1mPromptToken":150000,"percentageDiscount":0,"effectiveAt":"2025-09-12T08:38:04.98Z","endedAt":"","id":"27811d65-901f-4e0c-96c2-2b717aa3d824","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"8b9dceb5-a0cb-498f-9d2f-bab1f2451972","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.5:ray_fix_v1_otel_1.3.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-04T05:24:59.497516404Z","update_at":"2026-10-17T09:28:58.247041282Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 Next 80B A3B Thinking","artifact_description":"Qwen: Qwen3 Next 80B A3B Thinking","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-Next-80B-A3B-Thinking","artifact_template_id":"8b9dceb5-a0cb-498f-9d2f-bab1f2451972","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-Next-80B-A3B-Thinking","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3 Next 80B A3B Thinking\n\nQwen3 Next 80B A3B Thinking is a highly sparse MoE model with 80B total parameters but only ~3B activated per inference step. It excels at complex reasoning tasks, outperforming higher-cost models like Qwen3-30B-A3B-Thinking and Qwen3-32B-Thinking, and approaching the performance of the top-tier Qwen3-235B-A22B-Thinking.\n\n## API Usage\n\nYou can interact with the Qwen3 Next 80B A3B Thinking model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3 Next 80B A3B Thinking.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-Next-80B-A3B-Thinking\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-Next-80B-A3B-Thinking\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-Next-80B-A3B-Thinking","cacheHit":false,"pricePer1mPromptToken":150000,"pricePer1mCompletionToken":1500000,"originPricePer1mCompletionToken":1500000,"originPricePer1mPromptToken":150000,"percentageDiscount":0,"effectiveAt":"2025-09-12T08:38:04.98Z","endedAt":"","id":"6d9b481d-4590-43bb-bf26-96f1d6964dd6","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"a5c879a4-be0a-4621-95d3-42575238d9af","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.2:v1.5.15","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-09-11T23:58:37.460884671Z","update_at":"2026-10-27T06:45:31.643025124Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen3 Next 80B A3B Thinking","artifact_description":"Qwen: Qwen3 Next 80B A3B Thinking","artifact_tags":["LLM","Text-to-Text","Qwen3 Thinking"],"artifact_volume_path":"models/Qwen/Qwen3-Next-80B-A3B-Thinking","artifact_template_id":"a5c879a4-be0a-4621-95d3-42575238d9af","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"bf16","info_config_map":[{"key":"hf_link","value":"https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct","type":"text","min":0,"max":100,"step":0,"display_name":"Hugging Face","category":"feature"}],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-Next-80B-A3B-Thinking","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"10","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"5","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-Next-80B-A3B-Thinking","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-Next-80B-A3B-Thinking --tp-size 4 --mem-fraction-static 0.8 --trust-remote-code --tool-call-parser qwen25 --reasoning-parser deepseek-r1 --port 8000 --speculative-algo NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4","is_hidden":true}]},"artifact_details":{"model_description":"# Qwen3 Next 80B A3B Thinking\n\nQwen3 Next 80B A3B Thinking is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3 Next 80B A3B Thinking model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3 Next 80B A3B Thinking.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-Next-80B-A3B-Thinking\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-Next-80B-A3B-Thinking\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-Next-80B-A3B-Thinking","cacheHit":false,"pricePer1mPromptToken":150000,"pricePer1mCompletionToken":1500000,"originPricePer1mCompletionToken":1500000,"originPricePer1mPromptToken":150000,"percentageDiscount":0,"effectiveAt":"2025-09-12T08:38:04.98Z","endedAt":"","id":"6d9b481d-4590-43bb-bf26-96f1d6964dd6","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"434dc96d-11d9-4318-9ca4-556fa89eb7bb","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-20T03:15:07.000932258Z","update_at":"2026-10-17T09:28:58.435939616Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","artifact_description":"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","artifact_tags":["LLM"],"artifact_volume_path":"models/Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","artifact_template_id":"434dc96d-11d9-4318-9ca4-556fa89eb7bb","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3-235B-A22B-Instruct-2507-FP8\n\nQwen3-235B-A22B-Instruct-2507-FP8 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3-235B-A22B-Instruct-2507-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-235B-A22B-Instruct-2507-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-235B-A22B-Instruct-2507-FP8","cacheHit":false,"pricePer1mPromptToken":87500,"pricePer1mCompletionToken":350000,"originPricePer1mCompletionToken":1400000,"originPricePer1mPromptToken":350000,"percentageDiscount":7500,"effectiveAt":"2026-04-10T06:07:41.903599Z","endedAt":"","id":"db82e275-a21f-4730-b84d-99333d8e9e0b","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":87500,"outputPrice":350000,"cacheReadPrice":17500,"originInputPrice":350000,"originOutputPrice":1400000,"originCacheReadPrice":70000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"7b55b467-cf61-4b25-b65e-2903a7faa067","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-20T03:22:26.757461648Z","update_at":"2026-10-17T09:28:58.415887226Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","artifact_description":"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","artifact_tags":["LLM"],"artifact_volume_path":"models/Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","artifact_template_id":"7b55b467-cf61-4b25-b65e-2903a7faa067","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262141,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3-235B-A22B-Thinking-2507-FP8\n\nQwen3-235B-A22B-Thinking-2507-FP8 is a powerful large language model that has been distilled from larger models while maintaining strong performance. This model is designed to provide high-quality responses for a wide range of tasks while being more efficient than larger counterparts.\n\n## API Usage\n\nYou can interact with the Qwen3-235B-A22B-Thinking-2507-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-235B-A22B-Thinking-2507-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-235B-A22B-Thinking-2507-FP8","cacheHit":false,"pricePer1mPromptToken":350000,"pricePer1mCompletionToken":4200000,"originPricePer1mCompletionToken":4200000,"originPricePer1mPromptToken":350000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:08:47.039237Z","endedAt":"","id":"9bedb5b3-5ca3-4d1f-a5fa-a719aa276e8f","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"1fd8ead2-d150-4101-bc14-e46cd81d4a1e","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-01-20T03:09:35.042460286Z","update_at":"2026-10-17T09:28:58.443718776Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","artifact_description":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","artifact_tags":["LLM"],"artifact_volume_path":"models/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","artifact_template_id":"1fd8ead2-d150-4101-bc14-e46cd81d4a1e","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/openai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262128,"quantization":"fp8","info_config_map":[],"provider":"Qwen","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Qwen3-Coder-480B-A35B-Instruct-FP8\n\nQwen3-Coder is available in multiple sizes. Today, we're excited to introduce Qwen3-Coder-480B-A35B-Instruct-FP8. This streamlined model maintains impressive performance and efficiency, featuring the following key enhancements:\n\nSignificant Performance among open models on Agentic Coding, Agentic Browser-Use, and other foundational coding tasks.\nLong-context Capabilities with native support for 256K tokens, extendable up to 1M tokens using Yarn, optimized for repository-scale understanding.\nAgentic Coding supporting for most platform such as Qwen Code, CLINE, featuring a specially designed function call format.\n## API Usage\n\nYou can interact with the Qwen3-Coder-480B-A35B-Instruct-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-Coder-480B-A35B-Instruct-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","cacheHit":false,"pricePer1mPromptToken":900000,"pricePer1mCompletionToken":4500000,"originPricePer1mCompletionToken":4500000,"originPricePer1mPromptToken":900000,"percentageDiscount":0,"effectiveAt":"2026-04-10T06:28:05.196427Z","endedAt":"","id":"b841d686-b6ab-4461-8d31-63366a3a2cc2","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gemini","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":900000,"outputPrice":4500000,"cacheReadPrice":180000,"originInputPrice":900000,"originOutputPrice":4500000,"originCacheReadPrice":180000},{"threshold":32000,"inputPrice":1620000,"outputPrice":8100000,"cacheReadPrice":324000,"originInputPrice":1620000,"originOutputPrice":8100000,"originCacheReadPrice":324000},{"threshold":128000,"inputPrice":2700000,"outputPrice":13500000,"cacheReadPrice":540000,"originInputPrice":2700000,"originOutputPrice":13500000,"originCacheReadPrice":540000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"d4de48b8-2b40-49ae-85da-8f5f1016582a","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.6:ray_fix_otel_v9_qwen3_vl","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-12-11T14:27:15.304953524Z","update_at":"2026-10-27T06:45:31.302813434Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Qwen: Qwen3-VL-235B-A22B-Instruct-FP8","artifact_description":"Qwen/Qwen3-VL-235B-A22B-Instruct-FP8","artifact_tags":["Text-to-Text"],"artifact_volume_path":"models/Qwen/Qwen3-VL-235B-A22B-Instruct-FP8","artifact_template_id":"d4de48b8-2b40-49ae-85da-8f5f1016582a","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/qwen-ai-icon.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"fp8","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Qwen/Qwen3-VL-235B-A22B-Instruct-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"12","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"8","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Qwen/Qwen3-VL-235B-A22B-Instruct-FP8","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"Qwen/Qwen3-VL-235B-A22B-Instruct-FP8","is_hidden":true},{"key":"RATE_LIMIT","value":"1000000","is_hidden":true},{"key":"SGLANG_VLM_CACHE_SIZE_MB","value":"1024","is_hidden":true},{"key":"REQUEST_TIMEOUT","value":"5","is_hidden":true},{"key":"SGLANG_USE_CUDA_IPC_TRANSPORT","value":"1","is_hidden":true},{"key":"SGLANG_VALIDATE_IMAGE_URLS","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path Qwen/Qwen3-VL-235B-A22B-Instruct-FP8 --tp-size 4 --mem-fraction-static 0.8 --trust-remote-code --port 8000 --tool-call-parser qwen  --grammar-backend xgrammar --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 8}' --mm-process-config '{\"image\":{\"max_pixels\":2097152}}' --max-running-requests 24","is_hidden":true}]},"artifact_details":{"model_description":"# Qwen3-VL-235B-A22B-Instruct-FP8\n\nMeet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.\n\nThis generation delivers comprehensive upgrades across the board: superior text understanding \u0026 generation, deeper visual perception \u0026 reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities.\n\nAvailable in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on‑demand deployment.\n\n## Key Enhancements:\n- **Visual Agent**: Operates PC/mobile GUIs—recognizes elements, understands functions, invokes tools, completes tasks.\n\n- **Visual Coding Boost**: Generates Draw.io/HTML/CSS/JS from images/videos.\n\n- **Advanced Spatial Perception**: Judges object positions, viewpoints, and occlusions; provides stronger 2D grounding and enables 3D grounding for spatial reasoning and embodied AI.\n\n- **Long Context \u0026 Video Understanding**: Native 256K context, expandable to 1M; handles books and hours-long video with full recall and second-level indexing.\n\n- **Enhanced Multimodal Reasoning**: Excels in STEM/Math—causal analysis and logical, evidence-based answers.\n\n- **Upgraded Visual Recognition**: Broader, higher-quality pretraining is able to “recognize everything”—celebrities, anime, products, landmarks, flora/fauna, etc.\n\n- **Expanded OCR**: Supports 32 languages (up from 19); robust in low light, blur, and tilt; better with rare/ancient characters and jargon; improved long-document structure parsing.\n\n- **Text Understanding on par with pure LLMs**: Seamless text–vision fusion for lossless, unified comprehension.\n\n## API Usage\n\nYou can interact with the Qwen3-VL-235B-A22B-Instruct-FP8 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Qwen3-VL-235B-A22B-Instruct-FP8.\n\n### Shell\n```\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"Qwen/Qwen3-VL-235B-A22B-Instruct-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"Qwen/Qwen3-VL-235B-A22B-Instruct-FP8\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```"},"model_price":{"modelName":"Qwen/Qwen3-VL-235B-A22B-Instruct-FP8","cacheHit":false,"pricePer1mPromptToken":300000,"pricePer1mCompletionToken":1400000,"originPricePer1mCompletionToken":1400000,"originPricePer1mPromptToken":300000,"percentageDiscount":0,"effectiveAt":"2025-12-11T08:38:04.98Z","endedAt":"","id":"cb6c0f67-2951-4e69-81d5-cc2182fda895","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"abfc023e-35b7-4e9e-bb97-cef5db95f171","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.19:ray_fix_otel_v2","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-11-04T11:25:14.023947727Z","update_at":"2026-11-06T00:14:23.088095133Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ricursiveAI/Ripple-65B-Preview","artifact_description":"ricursiveAI/Ripple-65B-Preview","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/gmicloud/Ripple-65B-Preview","artifact_template_id":"abfc023e-35b7-4e9e-bb97-cef5db95f171","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/RI_icon_white_on_black.svg","preview_image_url":"","is_public":true,"model_info":{"context_length":1000000,"quantization":"bf16","info_config_map":[],"provider":"","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/gmicloud/Ripple-65B-Preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Ricursive AI / Ripple-65B-Preview\n\n## API Usage\n\nYou can interact with the ricursiveAI/Ripple-65B-Preview model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the ricursiveAI/Ripple-65B-Preview model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"gmicloud/Ripple-65B-Preview\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 1.0,\n    \"top_p\": 1.0,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"gmicloud/Ripple-65B-Preview\",\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"gmicloud/Ripple-65B-Preview","cacheHit":false,"pricePer1mPromptToken":200000,"pricePer1mCompletionToken":600000,"originPricePer1mCompletionToken":600000,"originPricePer1mPromptToken":200000,"percentageDiscount":0,"effectiveAt":"2026-11-04T18:20:31.72121Z","endedAt":"","id":"46f4dd16-9375-4e6b-8cc8-fd57f6c801dc","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":200000,"outputPrice":600000,"cacheReadPrice":50000,"originInputPrice":200000,"originOutputPrice":600000,"originCacheReadPrice":50000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"f2c4eeb4-8d4c-4b67-a4e5-82aba789412c","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-10-16T22:43:25.876935119Z","update_at":"2026-10-17T09:28:57.426111688Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"StepFun: Step-5-Preview","artifact_description":"StepFun: Step-5-Preview","artifact_tags":["LLM"],"artifact_volume_path":"models/stepfun-ai/Step-5-Preview","artifact_template_id":"f2c4eeb4-8d4c-4b67-a4e5-82aba789412c","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[],"provider":"Stepfun"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/stepfun-ai/Step-5-Preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.7,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Step-5-Preview\n\nStep 5 Preview is StepFun's flagship sparse Mixture-of-Experts (MoE) model for agentic work. It has 600B total parameters and activates about 27B parameters per token. It accepts text, image, and video input with text output, supports a 1M-token context window, and offers three reasoning effort levels (`low`, `medium`, `high`) to balance speed and cognitive depth. It is suited for repository-level software engineering, long-horizon tool use, multimodal analysis, and professional knowledge work, with particular strength in finance.\n\n## API Usage\n\nStep-5-Preview supports the OpenAI-compatible `/v1/chat/completions` endpoint. Set reasoning depth with `reasoning_effort` (`low`, `medium`, or `high`). Recommended sampling settings are `temperature=1.0` and `top_p=0.95`.\n\n## API Examples\n\n### OpenAI-Compatible Endpoint\n\nGenerate a model response using the chat completions endpoint.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"stepfun-ai/Step-5-Preview\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are an AI assistant provided by StepFun.\"},\n      {\"role\": \"user\", \"content\": \"Introduce StepFun'\\''s artificial intelligence capabilities.\"}\n    ],\n    \"reasoning_effort\": \"high\",\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer YOUR_API_KEY\"\n}\n\npayload = {\n    \"model\": \"stepfun-ai/Step-5-Preview\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant.\"},\n        {\"role\": \"user\", \"content\": \"Explain how sparse Mixture-of-Experts improves inference efficiency.\"}\n    ],\n    \"reasoning_effort\": \"high\",\n    \"temperature\": 1.0,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1024\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n#### Shell (Image Input)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"stepfun-ai/Step-5-Preview\",\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\"type\": \"text\", \"text\": \"What is in this picture?\"},\n          {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://example.com/photo.jpg\"}}\n        ]\n      }\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n\n#### Shell (Function Call)\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"stepfun-ai/Step-5-Preview\",\n    \"messages\": [\n      {\"role\": \"user\", \"content\": \"What is the weather in San Francisco?\"}\n    ],\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_weather\",\n          \"description\": \"Get weather in a location\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"location\": {\n                \"type\": \"string\",\n                \"description\": \"The city, e.g. San Francisco\"\n              }\n            },\n            \"required\": [\"location\"]\n          }\n        }\n      }\n    ],\n    \"max_tokens\": 1024\n  }'\n```\n"},"model_price":{"modelName":"stepfun-ai/Step-5-Preview","cacheHit":false,"pricePer1mPromptToken":1000000,"pricePer1mCompletionToken":2700000,"originPricePer1mCompletionToken":2700000,"originPricePer1mPromptToken":1000000,"percentageDiscount":0,"effectiveAt":"2026-10-16T23:46:08.655361Z","endedAt":"","id":"61d69236-1cb7-4f33-963d-146c5c9014d5","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1000000,"outputPrice":2700000,"cacheReadPrice":100000,"originInputPrice":1000000,"originOutputPrice":2700000,"originCacheReadPrice":100000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"3de77397-542f-49d5-830b-4a6c73811f88","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_0.11.2:otel_v5","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-11-27T13:09:56.19716719Z","update_at":"2026-10-27T06:45:31.343271543Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"Tencent: HunyuanOCR","artifact_description":"HunyuanOCR stands as a leading end-to-end OCR expert VLM powered by Hunyuan's native multimodal architecture. ","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/tencent/HunyuanOCR","artifact_template_id":"3de77397-542f-49d5-830b-4a6c73811f88","artifact_icon_link":"https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/Lp3m-XLpjQGwBItlvn69q.png","preview_image_url":"","is_public":true,"model_info":{"context_length":32768,"quantization":"bf16","info_config_map":[],"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/tencent/HunyuanOCR","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"40","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"48","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"32","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"tencent/HunyuanOCR","is_hidden":true},{"key":"gpu_scheduler","value":"true","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"SERVED_MODEL_NAME","value":"tencent/HunyuanOCR","is_hidden":true},{"key":"RAY_USAGE_STATS_ENABLED","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"10000000","is_hidden":true},{"key":"VLLM_ATTENTION_BACKEND","value":"FLASHINFER","is_hidden":true},{"key":"SERVE_COMMAND","value":"vllm serve tencent/HunyuanOCR  --no-enable-prefix-caching --mm-processor-cache-gb 0 --max-model-len 32768 --port 8000 --served-model-name tencent/HunyuanOCR","is_hidden":true}]},"artifact_details":{"model_description":"# HunyuanOCR \n\nHunyuanOCR stands as a leading end-to-end OCR expert VLM powered by Hunyuan's native multimodal architecture. With a remarkably lightweight 1B parameter design, it has achieved multiple state-of-the-art benchmarks across the industry. The model demonstrates mastery in complex multilingual document parsing while excelling in practical applications including text spotting, open-field information extraction, video subtitle extraction, and photo translation.\n\n## API Usage\n\nYou can interact with the HunyuanOCR model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of HunyuanOCR.\n\n### Shell\n```\ncurl https://api.gmi-serving.com/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer *************\" \\\n    -d '{\n  \"model\": \"tencent/HunyuanOCR\",\n  \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": [\n          {\n            \"type\": \"text\",\n            \"text\": \"What is in this image?\"\n          },\n          {\n            \"type\": \"image_url\",\n            \"image_url\": {\n              \"url\": \"https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png\"\n            }\n          }\n        ]\n      }\n    ],\n  \"temperature\": 0,\n  \"max_tokens\": 500\n}'\n```\n\n### Python\n```\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"tencent/HunyuanOCR\",\n    \"messages\": [\n        {\n          \"role\": \"user\",\n          \"content\": [\n            {\n              \"type\": \"text\",\n              \"text\": \"What is in this image?\"\n            },\n            {\n              \"type\": \"image_url\",\n              \"image_url\": {\n                \"url\": \"https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png\"\n              }\n            }\n          ]\n        }\n      ],\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n\n### Openai-python\n```\nimport os\nfrom openai import OpenAI\n\nclient = OpenAI(\n  base_url=\"https://api.gmi-serving.com/v1\",\n  api_key=\"\u003cGMI_API_KEY\u003e\",\n)\n\ncompletion = client.chat.completions.create(\n  model=\"tencent/HunyuanOCR\",\n  messages=[\n              {\n                \"role\": \"user\",\n                \"content\": [\n                  {\n                    \"type\": \"text\",\n                    \"text\": \"What is in this image? give me more details\"\n                  },\n                  {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                      \"url\": \"https://ofasys-multimodal-wlcb-3-toshanghai.oss-accelerate.aliyuncs.com/wpf272043/keepme/image/receipt.png\"\n                    }\n                  }\n                ]\n              }\n            ]\n)\n\nprint(completion.choices[0].message.content)\n```"},"model_price":{"modelName":"tencent/HunyuanOCR","cacheHit":false,"pricePer1mPromptToken":500000,"pricePer1mCompletionToken":500000,"originPricePer1mCompletionToken":500000,"originPricePer1mPromptToken":500000,"percentageDiscount":0,"effectiveAt":"2025-12-01T16:38:04.98Z","endedAt":"","id":"5c59edc2-4e35-4c28-9279-28976d59dccf","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"llm","series":"default","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"222715d2-d787-4feb-b5de-09e05a26bd35","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-05-11T19:24:51.108072336Z","update_at":"2026-10-17T09:28:58.061033478Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"tencent/hy3-preview","artifact_description":"tencent/hy3-preview","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/tencent/hy3-preview","artifact_template_id":"222715d2-d787-4feb-b5de-09e05a26bd35","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/tencent.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"bf16","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"Hunyuan","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/tencent/hy3-preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Tencent Hy3 Preview\n\nTencent Hy3 Preview is an open-source large language model developed by Tencent. It is designed for agentic workflows, coding, reasoning, long-context understanding, and tool-use scenarios. The model supports long-context processing with a 256K context window and is optimized for advanced AI applications such as coding assistants, agent orchestration, document analysis, and structured generation.\n\n## API Usage\n\nYou can interact with the Hy3 Preview model through a RESTful API using an OpenAI-compatible interface.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Hy3 Preview.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/hy3-preview\",\n    \"messages\": [\n      {\n        \"role\": \"system\",\n        \"content\": \"You are a helpful and capable AI assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Explain the concept of quantum entanglement in simple terms.\"\n      }\n    ],\n    \"temperature\": 0.7,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/hy3-preview\",\n    \"temperature\": 0.7,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"tencent/hy3-preview","cacheHit":false,"pricePer1mPromptToken":180000,"pricePer1mCompletionToken":600000,"originPricePer1mCompletionToken":600000,"originPricePer1mPromptToken":180000,"percentageDiscount":0,"effectiveAt":"2026-07-22T08:37:54.753277Z","endedAt":"","id":"0bacd9b5-4278-4467-b94e-d0ca98743a35","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":180000,"outputPrice":600000,"cacheReadPrice":60000,"originInputPrice":180000,"originOutputPrice":600000,"originCacheReadPrice":60000},{"threshold":16384,"inputPrice":240000,"outputPrice":960000,"cacheReadPrice":90000,"originInputPrice":240000,"originOutputPrice":960000,"originCacheReadPrice":90000},{"threshold":32768,"inputPrice":300000,"outputPrice":1200000,"cacheReadPrice":120000,"originInputPrice":300000,"originOutputPrice":1200000,"originCacheReadPrice":120000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"e3abda17-1c44-4632-a4ca-5430f487e6f1","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-11-04T04:29:08.351507145Z","update_at":"2026-11-04T07:29:09.299513233Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"tencent/hy4","artifact_description":"tencent/hy4","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/tencent/hy4","artifact_template_id":"e3abda17-1c44-4632-a4ca-5430f487e6f1","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/tencent-logo.png","preview_image_url":"","is_public":true,"model_info":{"context_length":1000000,"quantization":"bf16","info_config_map":[],"provider":"Hunyuan","is_open_source":false},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/tencent/hy4","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Tencent Hy4\n\nTencent Hy4 is a large language model developed by Tencent. It is designed for advanced agentic workflows, coding, complex reasoning, long-context understanding, and tool-use scenarios. The model is optimized for advanced AI applications such as coding assistants, agent orchestration, document analysis, structured generation, and complex multi-step task execution.\n\n## API Usage\n\nYou can interact with the Hy4 model through a RESTful API using an OpenAI-compatible interface.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Hy4.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/hy4\",\n    \"messages\": [\n      {\n        \"role\": \"system\",\n        \"content\": \"You are a helpful and capable AI assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Explain the concept of quantum entanglement in simple terms.\"\n      }\n    ],\n    \"temperature\": 0.7,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/hy4\",\n    \"temperature\": 0.7,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"tencent/hy4","cacheHit":false,"pricePer1mPromptToken":417000,"pricePer1mCompletionToken":1250500,"originPricePer1mCompletionToken":2501000,"originPricePer1mPromptToken":834000,"percentageDiscount":5000,"effectiveAt":"2026-11-04T05:05:37.33575Z","endedAt":"","id":"c5e6b380-5cbd-4289-b6bf-f30cc0c1202c","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":417000,"outputPrice":1250500,"cacheReadPrice":21000,"originInputPrice":834000,"originOutputPrice":2501000,"originCacheReadPrice":42000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"api-prod1.gmi-ie.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"d9b8429f-3610-4352-8ba1-d9c494ee03da","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/sglang_1gpu_prod:v1.1","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-27T20:28:03.514412374Z","update_at":"2026-10-17T09:28:57.763100837Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"tencent/hy4-preview","artifact_description":"tencent/hy4-preview","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/tencent/hy4-preview","artifact_template_id":"d9b8429f-3610-4352-8ba1-d9c494ee03da","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/tencent.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":262144,"quantization":"bf16","info_config_map":[],"provider":"Tencent"},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/tencent/hy4-preview","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_completion_tokens","value":4096,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Max Completion Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# Tencent Hy3 Preview\n\nTencent Hy3 Preview is an open-source large language model developed by Tencent. It is designed for agentic workflows, coding, reasoning, long-context understanding, and tool-use scenarios. The model supports long-context processing with a 256K context window and is optimized for advanced AI applications such as coding assistants, agent orchestration, document analysis, and structured generation.\n\n## API Usage\n\nYou can interact with the Hy3 Preview model through a RESTful API using an OpenAI-compatible interface.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of Hy3 Preview.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/hy3-preview\",\n    \"messages\": [\n      {\n        \"role\": \"system\",\n        \"content\": \"You are a helpful and capable AI assistant.\"\n      },\n      {\n        \"role\": \"user\",\n        \"content\": \"Explain the concept of quantum entanglement in simple terms.\"\n      }\n    ],\n    \"temperature\": 0.7,\n    \"top_p\": 0.95,\n    \"max_tokens\": 800\n  }'\n```\n\n### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"tencent/hy3-preview\",\n    \"temperature\": 0.7,\n    \"top_p\": 0.95,\n    \"max_tokens\": 1000,\n    \"tools\": [\n      {\n        \"type\": \"function\",\n        \"function\": {\n          \"name\": \"get_stock_price\",\n          \"description\": \"Retrieve the current stock price for a given company.\",\n          \"parameters\": {\n            \"type\": \"object\",\n            \"properties\": {\n              \"symbol\": {\n                \"type\": \"string\",\n                \"description\": \"Ticker symbol of the company, e.g. AAPL or TSLA.\"\n              }\n            },\n            \"required\": [\"symbol\"]\n          }\n        }\n      }\n    ],\n    \"messages\": [\n      {\n        \"role\": \"user\",\n        \"content\": \"What is the current price of Apple stock?\"\n      }\n    ]\n  }'\n```"},"model_price":{"modelName":"tencent/hy4-preview","cacheHit":false,"pricePer1mPromptToken":834000,"pricePer1mCompletionToken":2501000,"originPricePer1mCompletionToken":2501000,"originPricePer1mPromptToken":834000,"percentageDiscount":0,"effectiveAt":"2026-08-28T01:49:09.592155Z","endedAt":"","id":"e278b6b8-23e5-4dda-8381-808acf37a8ad","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":834000,"outputPrice":2501000,"cacheReadPrice":42000,"originInputPrice":834000,"originOutputPrice":2501000,"originCacheReadPrice":42000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"96b999fa-3c5c-430a-bfab-f92eab325c7b","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/wan22_lightning:v1.7.6_otel","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2025-12-05T06:55:05.654652711Z","update_at":"2026-10-27T06:45:31.309883992Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"wan2.2-lightning-I2V-8H100","artifact_description":"","artifact_tags":["Text-Image-to-Text"],"artifact_volume_path":"models/Wan-AI/Wan2.2-I2V-A14B","artifact_template_id":"96b999fa-3c5c-430a-bfab-f92eab325c7b","artifact_icon_link":"","preview_image_url":"","is_public":true,"model_info":{"context_length":2048,"quantization":"fp32","info_config_map":[],"provider":""},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/Wan-AI/Wan2.2-I2V-A14B","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[],"env_parameters":[{"key":"CKPT_DIR","value":"/mnt/model_garden","is_hidden":true},{"key":"LORA_DIR","value":"/mnt/lora","is_hidden":true},{"key":"WAN_WORLD_SIZE","value":"8","is_hidden":true},{"key":"WAN_STAGGER_DELAY","value":"5","is_hidden":true},{"key":"T5_FSDP","value":"false","is_hidden":true},{"key":"PARALLEL_VAE","value":"true","is_hidden":true},{"key":"FAST_VAE_ENCODE","value":"true","is_hidden":true},{"key":"VAE_FP16","value":"true","is_hidden":true},{"key":"TORCH_COMPILE","value":"true","is_hidden":true},{"key":"ASYNC_UPLOAD","value":"true","is_hidden":true},{"key":"ENABLE_WARMUP","value":"true","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"Wan2.2-lightningI2V","is_hidden":true},{"key":"SERVE_COMMAND","value":" ","is_hidden":true}]},"artifact_details":{"model_description":""},"model_price":{"modelName":"","cacheHit":false,"pricePer1mPromptToken":0,"pricePer1mCompletionToken":0,"originPricePer1mCompletionToken":0,"originPricePer1mPromptToken":0,"percentageDiscount":0,"effectiveAt":"","endedAt":"","id":"","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H100","gpuPrice":2980000,"billingType":"","series":"","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":null},"endpoints":[]},{"artifact_id":"7c24b593-2c99-4439-a9c4-fc4849e36760","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_glm5_h200:llm-engine_v3.4","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-04-07T00:26:14.760124402Z","update_at":"2026-10-27T06:45:31.153200843Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-5.1","artifact_description":"ZAI: GLM-5.1","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/zai-org/GLM-5.1-FP8","artifact_template_id":"7c24b593-2c99-4439-a9c4-fc4849e36760","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.1-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"4","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-5.1-FP8","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"previous_image","value":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_glm5_h200:llm-engine_v3.3","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SGLANG_ENABLE_SPEC_V2","value":"1","is_hidden":true},{"key":"SGLANG_NSA_FORCE_MLA","value":"1","is_hidden":true},{"key":"SGLANG_ENABLE_JIT_DEEPGEMM","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-5.1-FP8 --tokenizer-path /mnt/model_garden --tp-size 8 --trust-remote-code --tool-call-parser glm47 --reasoning-parser glm45 --attention-backend nsa --mem-fraction-static 0.9 --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --grammar-backend xgrammar --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}' --max-running-requests 16 --max-queued-requests 8 --enable-cache-report --port 8000","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-5.1\n\nGLM-5.1 is a cutting-edge large language model developed for high-performance natural language understanding and generation. It offers strong reasoning, code generation, and multilingual capabilities while maintaining excellent efficiency and cost-effectiveness. The model is optimized for general-purpose AI applications such as chat, content creation, programming assistance, and knowledge retrieval.\n\n\n## API Usage\n\nYou can interact with the GLM-5.1 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-5 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.1-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-5.1-FP8\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\n                        \"symbol\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```"},"model_price":{"modelName":"zai-org/GLM-5.1-FP8","cacheHit":false,"pricePer1mPromptToken":1400000,"pricePer1mCompletionToken":4400000,"originPricePer1mCompletionToken":4400000,"originPricePer1mPromptToken":1400000,"percentageDiscount":0,"effectiveAt":"2026-04-22T07:21:37.858745Z","endedAt":"","id":"db02aaa6-b894-425a-9cf5-7daf412813fa","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":1400000,"outputPrice":4400000,"cacheReadPrice":260000,"originInputPrice":1400000,"originOutputPrice":4400000,"originCacheReadPrice":260000}]},"endpoints":[]},{"artifact_id":"f2b6325c-ea70-4c19-802b-5a5f839259e3","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_vllm_k26_b200:llm-engine_v1.3","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-08-14T20:29:56.671487976Z","update_at":"2026-10-17T09:28:57.822078405Z","status":"ACTIVE","deployment_type":"llm_proxy"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-5.3","artifact_description":"ZAI: GLM-5.3","artifact_tags":["LLM","Text-to-Text"],"artifact_volume_path":"models/zai-org/GLM-5.3","artifact_template_id":"f2b6325c-ea70-4c19-802b-5a5f839259e3","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/kimi.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":1048576,"quantization":"fp8","info_config_map":[{"key":"supports_tool_calling","value":true,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Tool Calling","category":"general"},{"key":"supports_image_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Image Input","category":"general"},{"key":"supports_document_input","value":false,"type":"boolean","min":0,"max":0,"step":0,"display_name":"Supports Document Input","category":"general"}],"provider":"ZAI","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5.3","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":false},"artifact_parameters":{"model_parameters":[{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[]},"artifact_details":{"model_description":"# GLM-5.3\n\nGLM-5.3 is the latest flagship model in Z.ai's GLM-5 series, continuing the line's focus on coding and long-horizon agentic tasks. It uses the same base model as GLM-5.2, with all performance gains coming from scaled post-training on long-horizon task environments. Compared with GLM-5.2, GLM-5.3 delivers a 50% improvement on Z.ai Code Bench, achieves open-source SOTA on public benchmarks including Terminal Bench 3.0 and Agents' Last Exam, and shows emergent cyber capabilities for vulnerability discovery and exploitation reasoning.\n\nLike GLM-5.2, GLM-5.3 is designed to plan, execute, and iterate autonomously on extended, engineering-grade tasks. It supports a 1M-token context window and is optimized for complex software engineering, agentic workflows, tool use, and security analysis. GLM-5.3 supports three thinking effort levels ('low', 'high', 'max'); thinking is always enabled and cannot be disabled. For coding tasks, 'max' effort is recommended.\n\n## API Usage\n\nYou can interact with the GLM-5.3 model through various programming languages and methods. Below are examples showing how to use the model's API.\n\n## API Examples\n\nGenerate a model response using the chat endpoint of GLM-5.3.\n\n### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5.3\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n      {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"thinking\": {\"type\": \"enabled\"},\n    \"reasoning_effort\": \"max\",\n    \"temperature\": 0,\n    \"max_tokens\": 500\n  }'\n```\n\n#### example for function call\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-5.3\",\n    \"thinking\": {\"type\": \"enabled\"},\n    \"reasoning_effort\": \"max\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"query_weather\",\n                \"description\": \"Get weather of a city, the user should supply a city first\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"city\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city, e.g. Beijing\"\n                        }\n                    },\n                    \"required\": [\n                        \"city\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"How is the weather in Qingdao today?\"\n        }\n    ]\n}'\n```\n\n### Python\n\n```python\nimport requests\nimport json\n\nurl = \"https://api.gmi-serving.com/v1/chat/completions\"\nheaders = {\n    \"Content-Type\": \"application/json\",\n    \"Authorization\": \"Bearer *************\"\n}\n\npayload = {\n    \"model\": \"zai-org/GLM-5.3\",\n    \"messages\": [\n        {\"role\": \"system\", \"content\": \"You are a helpful AI assistant\"},\n        {\"role\": \"user\", \"content\": \"List 3 countries and their capitals.\"}\n    ],\n    \"thinking\": {\"type\": \"enabled\"},\n    \"reasoning_effort\": \"max\",\n    \"temperature\": 0,\n    \"max_tokens\": 500\n}\n\nresponse = requests.post(url, headers=headers, json=payload)\nprint(json.dumps(response.json(), indent=2))\n```\n"},"model_price":{"modelName":"zai-org/GLM-5.3","cacheHit":false,"pricePer1mPromptToken":700000,"pricePer1mCompletionToken":2200000,"originPricePer1mCompletionToken":4400000,"originPricePer1mPromptToken":1400000,"percentageDiscount":5000,"effectiveAt":"2026-08-14T21:20:42.43556Z","endedAt":"","id":"0e77bca1-594e-4a79-a57a-85ecec2c3170","valid":false,"gpuBillingUnit":"","gpuHardwareType":"","gpuPrice":0,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":700000,"outputPrice":2200000,"cacheReadPrice":130000,"originInputPrice":1400000,"originOutputPrice":4400000,"originCacheReadPrice":260000}]},"endpoints":[{"endpoint_status":"running","endpoint_url":"llm-proxy.gateway-prod-us.gmi-serving.com","dashboard_url":"","num_replicas":1}]},{"artifact_id":"863aa238-6023-481c-bb6a-5f0674675b12","artifact_data":{"artifact_type":"DockerImage","artifact_link":"us-west1-docker.pkg.dev/devv-404803/gmi-test-repo/gmi_sglang_0.5.8:glm-5-v7","artifact_resource":"from_default_template","build_status":"SUCCESS","build_error":"","build_id":"","build_file_name":"","create_at":"2026-03-12T20:57:37.71697726Z","update_at":"2026-10-27T06:45:31.181822506Z","status":"ACTIVE","deployment_type":"serverless"},"artifact_metadata":{"user_id":"","org_id":"","update_by":"","artifact_name":"ZAI: GLM-5 (H200)","artifact_description":"ZAI: GLM-5","artifact_tags":["LLM","Text-to-Text","GLM"],"artifact_volume_path":"models/zai-org/GLM-5-FP8","artifact_template_id":"863aa238-6023-481c-bb6a-5f0674675b12","artifact_icon_link":"https://storage.googleapis.com/gmi-artifact-metadata/shared_icons/zai.webp","preview_image_url":"","is_public":true,"model_info":{"context_length":202752,"quantization":"fp8","info_config_map":[],"provider":"ZAI","is_open_source":true},"is_featured":false,"model_source":{"type":"existing_path","volume_path":"models/zai-org/GLM-5-FP8","huggingface_model_id":"","huggingface_token":""},"available_on_maas":true,"available_on_dedicated":true},"artifact_parameters":{"model_parameters":[{"key":"temperature","value":0.5,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Temperature","category":"default","description":""},{"key":"max_tokens","value":4096,"type":"numeric","min":0,"max":16384,"step":1,"display_name":"Max Tokens","category":"default","description":""},{"key":"top_k","value":1,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top K","category":"default","description":""},{"key":"top_p","value":0.9,"type":"numeric","min":0,"max":1,"step":0.1,"display_name":"Top P","category":"default","description":""},{"key":"frequency_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Frequency Penalty","category":"default","description":""},{"key":"presence_penalty","value":0,"type":"numeric","min":-2,"max":2,"step":0.1,"display_name":"Presence Penalty","category":"default","description":""},{"key":"stream","value":true,"type":"boolean","min":0,"max":1,"step":0.1,"display_name":"Stream","category":"default","description":""},{"key":"system_prompt","value":"You are a helpful assistant","type":"text","min":0,"max":1,"step":0.1,"display_name":"System Prompt","category":"default","description":""}],"env_parameters":[{"key":"TARGET_ONGOING_REQUESTS","value":"8","is_hidden":true},{"key":"ENV","value":"prod","is_hidden":true},{"key":"MAX_ONGOING_REQUESTS","value":"16","is_hidden":true},{"key":"MAX_QUEUED_REQUESTS","value":"2","is_hidden":true},{"key":"KEY_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/api-keys/current","is_hidden":true},{"key":"TOKEN_VERIFICATION_URL","value":"https://console.gmicloud.ai/api/v1/me/profile","is_hidden":true},{"key":"PROMETHEUS_MULTIPROC_DIR","value":"/tmp/prometheus_multiproc","is_hidden":true},{"key":"OTEL_SERVICE_NAME","value":"zai-org/GLM-5-FP8","is_hidden":true},{"key":"RAY_LOG_TO_STDERR","value":"0","is_hidden":true},{"key":"RATE_LIMIT","value":"100000000","is_hidden":true},{"key":"HF_HUB_TRUST_REMOTE_CODE","value":"1","is_hidden":true},{"key":"SERVE_COMMAND","value":"python3 -m sglang.launch_server --model-path zai-org/GLM-5-FP8 --tp-size 8 --trust-remote-code --tool-call-parser glm47 --reasoning-parser glm45 --mem-fraction-static 0.9 --port 8000 --speculative-algorithm EAGLE --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --grammar-backend xgrammar --model-loader-extra-config '{\"enable_multithread_load\": true, \"num_threads\": 16}' --max-running-requests 16 --max-queued-requests 8","is_hidden":true}]},"artifact_details":{"model_description":"# GLM-5\n\nGLM-5 is a cutting-edge large language model developed for high-performance natural language understanding and generation. It offers strong reasoning, code generation, and multilingual capabilities while maintaining excellent efficiency and cost-effectiveness. The model is optimized for general-purpose AI applications such as chat, content creation, programming assistance, and knowledge retrieval.\n\n\n## API Usage\n\nYou can interact with the GLM-5 model through multiple programming environments using a RESTful API. The examples below demonstrate how to call the model for text generation and function-calling tasks.\n\n\n## API Examples\n\n### Generate a Chat Completion\n\nUse the chat completion endpoint to generate responses from the GLM-5 model.\n\n#### Shell\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"model\": \"zai-org/GLM-5-FP8\",\n    \"messages\": [\n      {\"role\": \"system\", \"content\": \"You are a knowledgeable AI assistant.\"},\n      {\"role\": \"user\", \"content\": \"Explain the concept of quantum entanglement in simple terms.\"}\n    ],\n    \"temperature\": 0.7,\n    \"max_tokens\": 800\n  }'\n```\n\n#### Function Call Example\n\n```bash\ncurl --request POST \\\n  --url https://api.gmi-serving.com/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer *************' \\\n  --data '{\n    \"temperature\": 0,\n    \"max_tokens\": 100,\n    \"model\": \"zai-org/GLM-5-FP8\",\n    \"tools\": [\n        {\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_stock_price\",\n                \"description\": \"Retrieve the current stock price for a given company.\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"symbol\": {\n                            \"type\": \"string\",\n                            \"description\": \"Ticker symbol of the company, e.g., AAPL or TSLA.\"\n                        }\n                    },\n                    \"required\": [\n                        \"symbol\"\n                    ]\n                }\n            }\n        }\n    ],\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"What is the current price of Apple stock?\"\n        }\n    ]\n}'\n```"},"model_price":{"modelName":"zai-org/GLM-5-FP8","cacheHit":false,"pricePer1mPromptToken":600000,"pricePer1mCompletionToken":1920000,"originPricePer1mCompletionToken":3200000,"originPricePer1mPromptToken":1000000,"percentageDiscount":4000,"effectiveAt":"2026-07-18T08:00:00Z","endedAt":"","id":"774f3f6e-f6ba-4784-b33d-ac8997fda039","valid":false,"gpuBillingUnit":"hourly","gpuHardwareType":"H200","gpuPrice":4500000,"billingType":"llm","series":"gpt","price":0,"originPrice":0,"chargeUnit":"","eligibleForPerksCredits":false,"eligibleForMinimaxCredits":false,"tiers":[{"threshold":0,"inputPrice":600000,"outputPrice":1920000,"cacheReadPrice":120000,"originInputPrice":1000000,"originOutputPrice":3200000,"originCacheReadPrice":200000}]},"endpoints":[]}]