Skip to content

Commit eb2c60e

Browse files
agentscope-ai-javaagentscope-ai-java
committed
docs: close remaining audit gaps, add realtime audio guide
1 parent c08de0c commit eb2c60e

9 files changed

Lines changed: 688 additions & 3 deletions

‎README.md‎

Lines changed: 111 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -350,6 +350,28 @@ response = MultiModalConversation.call(model="qwen-vl-max", messages=messages)
350350
print(response.output.choices[0].message.content[0]["text"])
351351
```
352352

353+
Pass `stream=True` to stream the response incrementally, same as `Generation`:
354+
355+
```python
356+
from dashscope import MultiModalConversation
357+
358+
messages = [{
359+
"role": "user",
360+
"content": [
361+
{"image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},
362+
{"text": "What does this picture describe?"},
363+
],
364+
}]
365+
responses = MultiModalConversation.call(
366+
model="qwen-vl-max",
367+
messages=messages,
368+
stream=True,
369+
incremental_output=True,
370+
)
371+
for response in responses:
372+
print(response.output.choices[0].message.content[0]["text"], end="")
373+
```
374+
353375
Use `AioMultiModalConversation` for the `async`/`await` form:
354376

355377
```python
@@ -460,13 +482,15 @@ from dashscope import MultiModalEmbedding
460482
from dashscope.embeddings.multimodal_embedding import (
461483
MultiModalEmbeddingItemText,
462484
MultiModalEmbeddingItemImage,
485+
MultiModalEmbeddingItemAudio,
463486
)
464487

465488
resp = MultiModalEmbedding.call(
466489
model="qwen3-vl-embedding",
467490
input=[
468491
MultiModalEmbeddingItemText(text="a red sports car", factor=1.0),
469492
MultiModalEmbeddingItemImage(image="https://dashscope.oss-cn-beijing.aliyuncs.com/images/256_1.png", factor=1.0),
493+
MultiModalEmbeddingItemAudio(audio="https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3", factor=1.0),
470494
],
471495
enable_fusion=True,
472496
)
@@ -578,6 +602,24 @@ if rsp.status_code == HTTPStatus.OK:
578602
print(result.url)
579603
```
580604

605+
`call` already blocks until the task completes; submit without blocking and poll separately with `async_call` + `wait`:
606+
607+
```python
608+
from dashscope import ImageSynthesis
609+
610+
task = ImageSynthesis.async_call(
611+
model="wanx2.1-t2i-turbo",
612+
prompt="a flower shop with delicate windows and a wooden door",
613+
n=1,
614+
size="1024*1024",
615+
)
616+
print(task.output.task_id)
617+
618+
rsp = ImageSynthesis.wait(task)
619+
for result in rsp.output.results:
620+
print(result.url)
621+
```
622+
581623
`sync_call` (currently only for `wan2.2-t2i-flash`/`wan2.2-t2i-plus`) returns the result directly instead of polling an async task:
582624

583625
```python
@@ -696,6 +738,31 @@ with open("output.wav", "wb") as f:
696738
f.write(result.get_audio_data())
697739
```
698740

741+
`SpeechSynthesisResult` also exposes sentence-level timestamps and the raw task response, useful for e.g. subtitle sync:
742+
743+
```python
744+
print(result.get_timestamps()) # per-sentence begin/end times
745+
print(result.get_response()) # the underlying SpeechSynthesisResponse (status, request_id, ...)
746+
```
747+
748+
For streaming instead of a single blocking call, subclass `ResultCallback` and pass it as `callback=`; `on_event` receives each `SpeechSynthesisResult` chunk as audio is generated:
749+
750+
```python
751+
from dashscope.audio.tts import SpeechSynthesizer, ResultCallback
752+
753+
class Callback(ResultCallback):
754+
def on_event(self, result) -> None:
755+
with open("output.wav", "ab") as f:
756+
f.write(result.get_audio_frame())
757+
758+
SpeechSynthesizer.call(
759+
model="cosyvoice-v1",
760+
text="Hello, Bailian.",
761+
format=SpeechSynthesizer.AudioFormat.format_wav,
762+
callback=Callback(),
763+
)
764+
```
765+
699766
`HttpSpeechSynthesizer` calls TTS over plain HTTP (no WebSocket), useful in environments that can't hold a persistent connection:
700767

701768
```python
@@ -785,6 +852,8 @@ with open("audio.pcm", "rb") as f:
785852
recognition.stop()
786853
```
787854

855+
For voice cloning, pronunciation fixes, real-time speech translation, and custom ASR hot words, see the [Advanced Audio guide](docs/guides/realtime-audio.md).
856+
788857
### Bailian Application (Agent App)
789858

790859
Call an app you built in [Bailian's Application Center](https://bailian.console.aliyun.com/):
@@ -841,6 +910,28 @@ print(len(tokens)) # token count
841910
print(tokenizer.decode(tokens)) # round-trip back to text
842911
```
843912

913+
`Tokenization.call` does the same job as a remote API call instead (useful for models not supported by the local tokenizer):
914+
915+
```python
916+
from dashscope import Tokenization
917+
918+
resp = Tokenization.call(model=Tokenization.Models.qwen_turbo, prompt="这个是千问tokenizer")
919+
print(resp.output["token_ids"], resp.output["tokens"])
920+
print(resp.usage["input_tokens"])
921+
```
922+
923+
### Listing Available Models
924+
925+
```python
926+
from dashscope import Models
927+
928+
models = Models.list(page=1, page_size=10)
929+
print(models.output["models"])
930+
931+
model = Models.get("qwen-plus")
932+
print(model.output["model_id"])
933+
```
934+
844935
### Fine-tuning
845936

846937
Upload a training file, launch a fine-tune job, and poll it to completion (requires `pip install "dashscope[rl]"` for agentic RL fine-tuning; classic supervised fine-tuning needs no extra):
@@ -898,8 +989,27 @@ dashscope generation create -m qwen-plus -p "Write a haiku about the sea" --stre
898989
dashscope models list
899990
dashscope models get qwen-plus
900991

901-
# Upload a file for fine-tuning
992+
# Upload, list, inspect, and delete files
902993
dashscope files upload -f ./train.jsonl -p fine_tune
994+
dashscope files list
995+
dashscope files get <file_id>
996+
dashscope files delete <file_id>
997+
998+
# Upload a file directly to OSS (used by some CV/vision models)
999+
dashscope oss upload -f ./photo.png -m wanx-style-repaint-v1
1000+
1001+
# Deploy a fine-tuned model, then manage the deployment
1002+
dashscope deployments create -m <finetuned-model-id> --plan mu -c 1
1003+
dashscope deployments list
1004+
dashscope deployments get <deployed_model>
1005+
dashscope deployments scale <deployed_model> -c 2
1006+
dashscope deployments delete <deployed_model>
1007+
1008+
# Agentic RL job management (see the reinforcement guide for `dashscope rl run`)
1009+
dashscope rl list
1010+
dashscope rl get <job_id>
1011+
dashscope rl logs <job_id>
1012+
dashscope rl cancel <job_id>
9031013
```
9041014

9051015
Run `dashscope --help` or `dashscope <command> --help` (e.g. `dashscope generation --help`) to see every command group (`generation`, `ft`, `files`, `deployments`, `models`, `embeddings`, `rerank`, `tokenization`, `application`, `image-synthesis`, `video-synthesis`, `multimodal-conversation`, `transcription`, `speech-synthesis`, `rl`, ...) and their options. Running `dashscope` with no arguments instead launches the interactive [AI Assistant](#ai-assistant-dashscope-sdk-expert).

‎README_ja.md‎

Lines changed: 111 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,28 @@ response = MultiModalConversation.call(model="qwen-vl-max", messages=messages)
343343
print(response.output.choices[0].message.content[0]["text"])
344344
```
345345

346+
`stream=True` を指定すると、`Generation` と同様に結果を逐次ストリーミングできます：
347+
348+
```python
349+
from dashscope import MultiModalConversation
350+
351+
messages = [{
352+
"role": "user",
353+
"content": [
354+
{"image": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20241022/emyrja/dog_and_girl.jpeg"},
355+
{"text": "この画像には何が写っていますか?"},
356+
],
357+
}]
358+
responses = MultiModalConversation.call(
359+
model="qwen-vl-max",
360+
messages=messages,
361+
stream=True,
362+
incremental_output=True,
363+
)
364+
for response in responses:
365+
print(response.output.choices[0].message.content[0]["text"], end="")
366+
```
367+
346368
`AioMultiModalConversation` を使うと `async`/`await` 形式で呼び出せます：
347369

348370
```python
@@ -453,13 +475,15 @@ from dashscope import MultiModalEmbedding
453475
from dashscope.embeddings.multimodal_embedding import (
454476
MultiModalEmbeddingItemText,
455477
MultiModalEmbeddingItemImage,
478+
MultiModalEmbeddingItemAudio,
456479
)
457480

458481
resp = MultiModalEmbedding.call(
459482
model="qwen3-vl-embedding",
460483
input=[
461484
MultiModalEmbeddingItemText(text="赤いスポーツカー", factor=1.0),
462485
MultiModalEmbeddingItemImage(image="https://dashscope.oss-cn-beijing.aliyuncs.com/images/256_1.png", factor=1.0),
486+
MultiModalEmbeddingItemAudio(audio="https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3", factor=1.0),
463487
],
464488
enable_fusion=True,
465489
)
@@ -571,6 +595,24 @@ if rsp.status_code == HTTPStatus.OK:
571595
print(result.url)
572596
```
573597

598+
`call` はそれ自体がタスク完了までブロックしますが、ブロックせずに送信して `async_call` + `wait` で別途ポーリングすることもできます：
599+
600+
```python
601+
from dashscope import ImageSynthesis
602+
603+
task = ImageSynthesis.async_call(
604+
model="wanx2.1-t2i-turbo",
605+
prompt="繊細な窓と木製の扉がある花屋",
606+
n=1,
607+
size="1024*1024",
608+
)
609+
print(task.output.task_id)
610+
611+
rsp = ImageSynthesis.wait(task)
612+
for result in rsp.output.results:
613+
print(result.url)
614+
```
615+
574616
`sync_call`（現在は `wan2.2-t2i-flash`/`wan2.2-t2i-plus` のみ対応）は、非同期タスクをポーリングする代わりに結果を直接返します：
575617

576618
```python
@@ -689,6 +731,31 @@ with open("output.wav", "wb") as f:
689731
f.write(result.get_audio_data())
690732
```
691733

734+
`SpeechSynthesisResult` は、文単位のタイムスタンプや元のタスクレスポンスも公開しており、字幕同期などに利用できます：
735+
736+
```python
737+
print(result.get_timestamps()) # 各文の開始・終了時刻
738+
print(result.get_response()) # 内部の SpeechSynthesisResponse（status、request_id など）
739+
```
740+
741+
1回のブロッキング呼び出しではなくストリーミングにしたい場合は、`ResultCallback` を継承して `callback=` に渡します。音声が生成されるたびに `on_event` が各 `SpeechSynthesisResult` チャンクを受け取ります：
742+
743+
```python
744+
from dashscope.audio.tts import SpeechSynthesizer, ResultCallback
745+
746+
class Callback(ResultCallback):
747+
def on_event(self, result) -> None:
748+
with open("output.wav", "ab") as f:
749+
f.write(result.get_audio_frame())
750+
751+
SpeechSynthesizer.call(
752+
model="cosyvoice-v1",
753+
text="Hello, Bailian.",
754+
format=SpeechSynthesizer.AudioFormat.format_wav,
755+
callback=Callback(),
756+
)
757+
```
758+
692759
`HttpSpeechSynthesizer` は通常の HTTP（WebSocket 不要）で音声合成を呼び出します。持続的な接続を維持できない環境で便利です：
693760

694761
```python
@@ -778,6 +845,8 @@ with open("audio.pcm", "rb") as f:
778845
recognition.stop()
779846
```
780847

848+
音声クローン、発音修正、リアルタイム音声翻訳、カスタム ASR ホットワードについては、[高度な音声機能ガイド](docs/guides/realtime-audio_ja.md)を参照してください。
849+
781850
### 百煉アプリケーション（エージェントアプリ）
782851

783852
[百煉のアプリケーションセンター](https://bailian.console.aliyun.com/) で作成したアプリを呼び出します。
@@ -834,6 +903,28 @@ print(len(tokens)) # トークン数
834903
print(tokenizer.decode(tokens)) # トークンからテキストへ復元
835904
```
836905

906+
`Tokenization.call` はリモート API 呼び出しで同じ処理を行います（ローカルトークナイザーが対応していないモデルに便利です）：
907+
908+
```python
909+
from dashscope import Tokenization
910+
911+
resp = Tokenization.call(model=Tokenization.Models.qwen_turbo, prompt="这个是千问tokenizer")
912+
print(resp.output["token_ids"], resp.output["tokens"])
913+
print(resp.usage["input_tokens"])
914+
```
915+
916+
### 利用可能なモデルの一覧表示
917+
918+
```python
919+
from dashscope import Models
920+
921+
models = Models.list(page=1, page_size=10)
922+
print(models.output["models"])
923+
924+
model = Models.get("qwen-plus")
925+
print(model.output["model_id"])
926+
```
927+
837928
### ファインチューニング
838929

839930
学習用ファイルをアップロードし、ファインチューニングジョブを作成して完了まで待機します（Agentic RL ファインチューニングには `pip install "dashscope[rl]"` が必要。従来の教師ありファインチューニングには追加インストール不要）：
@@ -891,8 +982,27 @@ dashscope generation create -m qwen-plus -p "海についての俳句を書い
891982
dashscope models list
892983
dashscope models get qwen-plus
893984

894-
# ファインチューニング用ファイルのアップロード
985+
# ファイルのアップロード、一覧表示、詳細確認、削除
895986
dashscope files upload -f ./train.jsonl -p fine_tune
987+
dashscope files list
988+
dashscope files get <file_id>
989+
dashscope files delete <file_id>
990+
991+
# OSS へ直接ファイルをアップロード（一部の CV/ビジョンモデルで使用）
992+
dashscope oss upload -f ./photo.png -m wanx-style-repaint-v1
993+
994+
# ファインチューニング済みモデルをデプロイし、そのデプロイメントを管理する
995+
dashscope deployments create -m <finetuned-model-id> --plan mu -c 1
996+
dashscope deployments list
997+
dashscope deployments get <deployed_model>
998+
dashscope deployments scale <deployed_model> -c 2
999+
dashscope deployments delete <deployed_model>
1000+
1001+
# Agentic RL ジョブの管理（`dashscope rl run` の使い方は強化学習ガイドを参照）
1002+
dashscope rl list
1003+
dashscope rl get <job_id>
1004+
dashscope rl logs <job_id>
1005+
dashscope rl cancel <job_id>
8961006
```
8971007

8981008
`dashscope --help` または `dashscope <command> --help`（例：`dashscope generation --help`）を実行すると、すべてのコマンドグループ（`generation`、`ft`、`files`、`deployments`、`models`、`embeddings`、`rerank`、`tokenization`、`application`、`image-synthesis`、`video-synthesis`、`multimodal-conversation`、`transcription`、`speech-synthesis`、`rl` など）とそのオプションを確認できます。引数なしで `dashscope` を実行すると、対話型の [AI アシスタント](#ai-アシスタントdashscope-sdk-expert)が起動します。

0 commit comments

Comments
 (0)