Hugging Face's logo

freddm
/
JoyAI-LLM-Flash-GGUF

Text Generation
GGUF
Chinese
English
conversational
Model card Files
xet
Community
freddm commited on
Commit
89fbc63
·
verified ·
1 Parent(s): 0cc0fa4

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +380 -0
README.md ADDED
@@ -0,0 +1,380 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zh
4
+ - en
5
+ pipeline_tag: text-generation
6
+ ---
7
+ <div align="center">
8
+ <picture>
9
+ <img src="figures/joyai-logo.png" width="30%" alt="JoyAI-LLM Flash">
10
+ </picture>
11
+ </div>
12
+ <hr>
13
+
14
+ <div align="center" style="line-height: 1;">
15
+ <a href="https://huggingface.co/jdopensource" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-JD-ffc107?color=ffc107&logoColor=white"/></a>
16
+ <a href="https://huggingface.co/jdopensource/JoyAI-LLM-Flash/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a>
17
+ </div>
18
+
19
+
20
+
21
+
22
+ ## 1. Model Introduction
23
+
24
+ JoyAI-LLM Flash is a state-of-the-art medium-sized instruct language model with 3 billion activated parameters and 48 billion total parameters. JoyAI-LLM Flash was pretrained on 20 trillion text tokens using Muon optimizer, followed by large-scale supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (RL) across diverse environments. JoyAI-LLM Flash achieves strong performance across frontier knowledge, reasoning, coding tasks and agentic capabilities.
25
+
26
+ ### Key Features
27
+
28
+ - Fiber Bundle RL: Introduces fiber bundle theory into reinforcement learning, proposing a novel optimization framework, FiberPO. This method is specifically designed to handle the challenges of large-scale and heterogeneous agent training, improving stability and robustness under complex data distributions.
29
+ - Training-Inference Collaboration: apply Muon optimizer with dense MTP, develop novel optimization techniques to resolve instabilities while scaling up, delivering 1.3× to 1.7× the throughput of the non-MTP version.
30
+ - Agentic Intelligence: designed for tool use, reasoning, and autonomous problem-solving.
31
+
32
+ ## 2. Model Summary
33
+
34
+ | | |
35
+ | :-----------------------------------------: | :----------------------: |
36
+ | **Architecture** | Mixture-of-Experts (MoE) |
37
+ | **Total Parameters** | 48B |
38
+ | **Activated Parameters** | 3B |
39
+ | **Number of Layers** (Dense layer included) | 40 |
40
+ | **Number of Dense Layers** | 1 |
41
+ | **Attention Hidden Dimension** | 2048 |
42
+ | **MoE Hidden Dimension** (per Expert) | 768 |
43
+ | **Number of Attention Heads** | 32 |
44
+ | **Number of Experts** | 256 |
45
+ | **Selected Experts per Token** | 8 |
46
+ | **Number of Shared Experts** | 1 |
47
+ | **Vocabulary Size** | 129K |
48
+ | **Context Length** | 128K |
49
+ | **Attention Mechanism** | MLA |
50
+ | **Activation Function** | SwiGLU |
51
+ | </div> | |
52
+
53
+
54
+ ## 3. Evaluation Results
55
+
56
+ <table>
57
+ <thead>
58
+ <tr>
59
+ <th align="center">Benchmark</th>
60
+ <th align="center"><sup>JoyAI-LLM Flash</sup></th>
61
+ <th align="center"><sup>Qwen3-30B-A3B-Instuct-2507</sup></th>
62
+ <th align="center"><sup>GLM-4.7-Flash<br>(Non-thinking)</sup></th>
63
+ </tr>
64
+ </thead>
65
+ <tbody>
66
+
67
+
68
+ <tr>
69
+ <td align="center" colspan=8><strong>Knowledge &amp; Alignment</strong></td>
70
+ </tr>
71
+ <tr>
72
+ <td align="center" style="vertical-align: middle">MMLU</td>
73
+ <td align="center" style="vertical-align: middle"><strong>89.50</strong></td>
74
+ <td align="center" style="vertical-align: middle">86.87</td>
75
+ <td align="center" style="vertical-align: middle">80.53</td>
76
+ </tr>
77
+ <tr>
78
+ <td align="center" style="vertical-align: middle">MMLU-Pro</td>
79
+ <td align="center" style="vertical-align: middle"><strong>81.02</strong></td>
80
+ <td align="center" style="vertical-align: middle">73.88</td>
81
+ <td align="center" style="vertical-align: middle">63.62</td>
82
+ </tr>
83
+ <tr>
84
+ <td align="center" style="vertical-align: middle">CMMLU</td>
85
+ <td align="center" style="vertical-align: middle"><strong>87.03</strong></td>
86
+ <td align="center" style="vertical-align: middle">85.88</td>
87
+ <td align="center" style="vertical-align: middle">75.85</td>
88
+ </tr>
89
+ <tr>
90
+ <td align="center" style="vertical-align: middle">GPQA-Diamond</td>
91
+ <td align="center" style="vertical-align: middle"><strong>74.43</strong></td>
92
+ <td align="center" style="vertical-align: middle">68.69</td>
93
+ <td align="center" style="vertical-align: middle">39.90</td>
94
+ </tr>
95
+ <tr>
96
+ <td align="center" style="vertical-align: middle">SuperGPQA</td>
97
+ <td align="center" style="vertical-align: middle"><strong>55.00</strong></td>
98
+ <td align="center" style="vertical-align: middle">52.00</td>
99
+ <td align="center" style="vertical-align: middle">32.00</td>
100
+ </tr>
101
+ <tr>
102
+ <td align="center" style="vertical-align: middle">LiveBench</td>
103
+ <td align="center" style="vertical-align: middle"><strong>72.90</strong></td>
104
+ <td align="center" style="vertical-align: middle">59.70</td>
105
+ <td align="center" style="vertical-align: middle">43.10</td>
106
+ </tr>
107
+ <tr>
108
+ <td align="center" style="vertical-align: middle">IFEval</td>
109
+ <td align="center" style="vertical-align: middle"><strong>86.69</strong></td>
110
+ <td align="center" style="vertical-align: middle">83.18</td>
111
+ <td align="center" style="vertical-align: middle">82.44</td>
112
+ </tr>
113
+ <tr>
114
+ <td align="center" style="vertical-align: middle">AlignBench</td>
115
+ <td align="center" style="vertical-align: middle"><strong>8.24</strong></td>
116
+ <td align="center" style="vertical-align: middle">8.07</td>
117
+ <td align="center" style="vertical-align: middle">6.85</td>
118
+ </tr>
119
+ <tr>
120
+ <td align="center" style="vertical-align: middle">HellaSwag</td>
121
+ <td align="center" style="vertical-align: middle"><strong>91.79</strong></td>
122
+ <td align="center" style="vertical-align: middle">89.90</td>
123
+ <td align="center" style="vertical-align: middle">60.84</td>
124
+ </tr>
125
+
126
+ <tr>
127
+ <td align="center" colspan=8><strong>Coding</strong></td>
128
+ </tr>
129
+ <tr>
130
+ <td align="center" style="vertical-align: middle">HumanEval</td>
131
+ <td align="center" style="vertical-align: middle"><strong>96.34</strong></td>
132
+ <td align="center" style="vertical-align: middle">95.12</td>
133
+ <td align="center" style="vertical-align: middle">74.39</td>
134
+ </tr>
135
+ <tr>
136
+ <td align="center" style="vertical-align: middle">LiveCodeBench</td>
137
+ <td align="center" style="vertical-align: middle"><strong>65.60</strong></td>
138
+ <td align="center" style="vertical-align: middle">39.71</td>
139
+ <td align="center" style="vertical-align: middle">27.43</td>
140
+ </tr>
141
+ <tr>
142
+ <td align="center" style="vertical-align: middle">SciCode</td>
143
+ <td align="center" style="vertical-align: middle"><strong>3.08/22.92</strong></td>
144
+ <td align="center" style="vertical-align: middle"><strong>3.08/22.92</strong></td>
145
+ <td align="center" style="vertical-align: middle">3.08/15.11</td>
146
+ </tr>
147
+ <tr>
148
+ <td align="center" colspan=8><strong>Mathematics</strong></td>
149
+ </tr>
150
+ <tr>
151
+ <td align="center" style="vertical-align: middle">GSM8K</td>
152
+ <td align="center" style="vertical-align: middle"><strong>95.83</strong></td>
153
+ <td align="center" style="vertical-align: middle">79.83</td>
154
+ <td align="center" style="vertical-align: middle">81.88</td>
155
+ </tr>
156
+ <tr>
157
+ <td align="center" style="vertical-align: middle">AIME2025</td>
158
+ <td align="center" style="vertical-align: middle"><strong>65.83</strong></td>
159
+ <td align="center" style="vertical-align: middle">62.08</td>
160
+ <td align="center" style="vertical-align: middle">24.17</td>
161
+ </tr>
162
+ <tr>
163
+ <td align="center" style="vertical-align: middle">MATH 500</td>
164
+ <td align="center" style="vertical-align: middle"><strong>97.10</strong></td>
165
+ <td align="center" style="vertical-align: middle">89.80</td>
166
+ <td align="center" style="vertical-align: middle">90.90</td>
167
+ </tr>
168
+
169
+ <tr>
170
+ <td align="center" colspan=8><strong>Agentic</strong></td>
171
+ </tr>
172
+ <tr>
173
+ <td align="center" style="vertical-align: middle">SWE-bench Verified</td>
174
+ <td align="center" style="vertical-align: middle"><strong>60.60</strong></td>
175
+ <td align="center" style="vertical-align: middle">24.44</td>
176
+ <td align="center" style="vertical-align: middle">51.60</td>
177
+ </tr>
178
+ <tr>
179
+ <td align="center" style="vertical-align: middle">Tau2-Retail</td>
180
+ <td align="center" style="vertical-align: middle"><strong>67.55</strong></td>
181
+ <td align="center" style="vertical-align: middle">53.51</td>
182
+ <td align="center" style="vertical-align: middle">62.28</td>
183
+ </tr>
184
+ <tr>
185
+ <td align="center" style="vertical-align: middle">Tau2-Airline</td>
186
+ <td align="center" style="vertical-align: middle"><strong>54.00</strong></td>
187
+ <td align="center" style="vertical-align: middle">32.00</td>
188
+ <td align="center" style="vertical-align: middle">52.00</td>
189
+ </tr>
190
+ <tr>
191
+ <td align="center" style="vertical-align: middle">Tau2-Telecom</td>
192
+ <td align="center" style="vertical-align: middle">79.83</td>
193
+ <td align="center" style="vertical-align: middle">4.39</td>
194
+ <td align="center" style="vertical-align: middle"><strong>88.60</strong></td>
195
+ </tr>
196
+
197
+ <tr>
198
+ <td align="center" colspan=8><strong>Long Context</strong></td>
199
+ </tr>
200
+ <tr>
201
+ <td align="center" style="vertical-align: middle">RULER</td>
202
+ <td align="center" style="vertical-align: middle"><strong>95.60</strong></td>
203
+ <td align="center" style="vertical-align: middle">89.66</td>
204
+ <td align="center" style="vertical-align: middle">56.12</td>
205
+ </tr>
206
+ </tbody>
207
+ </table>
208
+
209
+
210
+ ## 4. Deployment
211
+
212
+ > [!Note]
213
+ > You can access JoyAI-LLM Flash API on https://docs.jdcloud.com/cn/jdaip/chat and we provide OpenAI/Anthropic-compatible API for you.
214
+ > Currently, JoyAI-LLM Flash is recommended to run on the following inference engines:
215
+
216
+ * vLLM
217
+ * SGLang
218
+
219
+ The minimum version requirement for `transformers` is `4.57.1`.
220
+
221
+ Deployment examples can be found in the [Model Deployment Guide](docs/deploy_guidance.md).
222
+
223
+
224
+
225
+ ## 5. Model Usage
226
+
227
+ The usage demos below demonstrate how to call our official API.
228
+
229
+ For third-party APIs deployed with vLLM or SGLang, please note that:
230
+
231
+ > [!Note] Recommended sampling parameters: `temperature=0.6`, `top_p=1.0`
232
+
233
+ ### Chat Completion
234
+
235
+ This is a simple chat completion script which shows how to call JoyAI-Flash API.
236
+
237
+ ```python
238
+ from openai import OpenAI
239
+
240
+ client = OpenAI(base_url="http://IP:PORT/v1", api_key="EMPTY")
241
+
242
+
243
+ def simple_chat(client: OpenAI):
244
+ messages = [
245
+ {
246
+ "role": "user",
247
+ "content": [
248
+ {
249
+ "type": "text",
250
+ "text": "which one is bigger, 9.11 or 9.9? think carefully.",
251
+ }
252
+ ],
253
+ },
254
+ ]
255
+ model_name = client.models.list().data[0].id
256
+ response = client.chat.completions.create(
257
+ model=model_name, messages=messages, stream=False, max_tokens=4096
258
+ )
259
+ print(f"response: {response.choices[0].message.content}")
260
+
261
+
262
+ if __name__ == "__main__":
263
+ simple_chat(client)
264
+ ```
265
+
266
+
267
+ ### Tool call Completion
268
+
269
+ This is a simple toll call completion script which shows how to call JoyAI-Flash API.
270
+
271
+ ```python
272
+ import json
273
+
274
+ from openai import OpenAI
275
+
276
+ client = OpenAI(base_url="http://IP:PORT/v1", api_key="EMPTY")
277
+
278
+
279
+ def my_calculator(expression: str) -> str:
280
+ return str(eval(expression))
281
+
282
+
283
+ def rewrite(expression: str) -> str:
284
+ return str(expression)
285
+
286
+
287
+ def simple_tool_call(client: OpenAI):
288
+ messages = [
289
+ {
290
+ "role": "user",
291
+ "content": [
292
+ {
293
+ "type": "text",
294
+ "text": "use my functions to compute the results for the equations: 6+1",
295
+ },
296
+ ],
297
+ },
298
+ ]
299
+ tools = [
300
+ {
301
+ "type": "function",
302
+ "function": {
303
+ "name": "my_calculator",
304
+ "description": "A calculator that can evaluate a mathematical equation and compute its results.",
305
+ "parameters": {
306
+ "type": "object",
307
+ "properties": {
308
+ "expression": {
309
+ "type": "string",
310
+ "description": "The mathematical expression to evaluate.",
311
+ },
312
+ },
313
+ "required": ["expression"],
314
+ },
315
+ },
316
+ },
317
+ {
318
+ "type": "function",
319
+ "function": {
320
+ "name": "rewrite",
321
+ "description": "Rewrite a given text for improved clarity",
322
+ "parameters": {
323
+ "type": "object",
324
+ "properties": {
325
+ "text": {
326
+ "type": "string",
327
+ "description": "The input text to rewrite",
328
+ }
329
+ },
330
+ },
331
+ },
332
+ },
333
+ ]
334
+ model_name = client.models.list().data[0].id
335
+ response = client.chat.completions.create(
336
+ model=model_name,
337
+ messages=messages,
338
+ temperature=1.0,
339
+ max_tokens=1024,
340
+ tools=tools,
341
+ tool_choice="auto",
342
+ )
343
+ tool_calls = response.choices[0].message.tool_calls
344
+
345
+ results = []
346
+ for tool_call in tool_calls:
347
+ function_name = tool_call.function.name
348
+ function_args = tool_call.function.arguments
349
+ if function_name == "my_calculator":
350
+ result = my_calculator(**json.loads(function_args))
351
+ results.append(result)
352
+ messages.append({"role": "assistant", "tool_calls": tool_calls})
353
+ for tool_call, result in zip(tool_calls, results):
354
+ messages.append(
355
+ {
356
+ "role": "tool",
357
+ "tool_call_id": tool_call.id,
358
+ "name": tool_call.function.name,
359
+ "content": result,
360
+ }
361
+ )
362
+ response = client.chat.completions.create(
363
+ model=model_name,
364
+ messages=messages,
365
+ temperature=1.0,
366
+ max_tokens=1024,
367
+ )
368
+ print(response.choices[0].message.content)
369
+
370
+
371
+ if __name__ == "__main__":
372
+ simple_tool_call(client)
373
+
374
+ ```
375
+
376
+ ---
377
+
378
+ ## 6. License
379
+
380
+ Both the code repository and the model weights are released under the [Modified MIT License](LICENSE).