qwen-omni example does not work on the Mac

[scalar27]

I'm on an M1 Mac, and tried running the qwen-omni example but there's an architecture mismatch. It fails with this error:

1 warning found (use docker --debug to expand):

• InvalidBaseImagePlatform: Base image vllm/vllm-openai:v0.10.0 was pulled with platform "linux/amd64", expected "linux/arm64" for current build (line 4)
  WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
  docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

Please suggest a workaround if there is one. Thanks!

[keepingitneil]

Oh nice find. I'll make a fix ASAP

[keepingitneil]

Ok I have introduced llama.cpp to the repo for running qwen on mac. I've made relevant changes to the README and tested on my M2 16GB Macbook Pro #77

[scalar27]

I've pulled the latest changes. I'm getting popup errors: "Failed to start run. Please try again" and no voice output.
Steps: I cd to local-llm, and run ./qwen-omni-7b-q4km_llamacpp.sh (this works fine, I can check that llama-server is functioning properly). I open a second shell and run "docker compose up" in the main directory. I then go to localhost:3000 and start the example "local conversational with vision". The workflow graphics appear and I allow mic/camera/screen sharing and hit run. After a while I get the errors "Failed to start run. Please try again". Suggestions to fix? Thanks.

[keepingitneil]

I'll try and reproduce. Do you see any error logs in the terminal that you ran docker compose in?

[scalar27]

A ton of output is produced. Here are some lines that look like a problem: repository-1 | ERROR:asyncio:Unclosed client session
repository-1 | client_session: <aiohttp.client.ClientSession object at 0xffff58766150>
repository-1 | ERROR:asyncio:Unclosed connector
repository-1 | connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0xffff5812c8f0>, 350.624212033)])']
repository-1 | connector: <aiohttp.connector.TCPConnector object at 0xffff58766210>
repository-1 | INFO:aiohttp.access:162.196.80.37 [20/Aug/2025:06:04:56 +0000] "POST /app/run HTTP/1.1" 200 771 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3.1 Safari/605.1.15"
engine-1 | 2025-08-20 06:04:56,442 - DEBUG root - pending request count: 1 {"pid": 406, "job_id": "AJ_Bmoq2bApv49Y"}
livekit-1 | 2025-08-20T06:04:56.586Z INFO livekit agent/client.go:173 failed to send job request {"error": "no servers available (received 1 responses)", "namespace": "", "jobType": "JT_ROOM", "agentName": "gabber-engine"}
livekit-1 | 2025-08-20T06:04:57.385Z INFO livekit agent/client.go:173 failed to send job request {"error": "no response from servers", "namespace": "", "jobType": "JT_ROOM", "agentName": ""}

[scalar27]

more:
engine-1 | 2025-08-20 06:17:37,066 - INFO livekit.agents - process exiting {"reason": "", "pid": 35, "job_id": "AJ_MTVCKLiKH3VS"}
engine-1 | 2025-08-20 06:17:37,066 - DEBUG livekit.agents - shutting down job task {"reason": "", "user_initiated": false, "pid": 35, "job_id": "AJ_MTVCKLiKH3VS"}
engine-1 | 2025-08-20 06:17:37,068 - WARNING livekit - livekit::rtc_engine:453:livekit::rtc_engine - received session close: "signal client closed: "stream closed"" UnknownReason Resume {"pid": 35, "job_id": "AJ_MTVCKLiKH3VS"}

[scalar27]

engine-1 | 2025-08-20 06:17:49,892 - DEBUG root - pending request count: 1 {"pid": 49, "job_id": "AJ_TDGbN5Nj3iob"}
engine-1 | 2025-08-20 06:17:50,893 - DEBUG root - pending request count: 1 {"pid": 49, "job_id": "AJ_TDGbN5Nj3iob"}
engine-1 | 2025-08-20 06:17:51,895 - WARNING root - RequestContext <core.pad.request_context.RequestContext object at 0xffff5c415550> timed out after 30.06 seconds. Originator: None {"pid": 49, "job_id": "AJ_TDGbN5Nj3iob"}
engine-1 | 2025-08-20 06:17:51,896 - WARNING root - RequestContext timed out. {"pid": 49, "job_id": "AJ_TDGbN5Nj3iob"}

[keepingitneil]

Appreciate you trying this. Yeah the logs are noisy (I need to clean them up a bit) but I don't see anything too alarming.

Can you test the basic hello world echo example?

[scalar27]

I'm still not sure what the hello world is supposed to do, but I don't think it's working either. Do I need to click all three icons (camera, mic, screen share) and then hit run? I also get a bunch of console errors or warnings.

[scalar27]

and these errors:
"POST /app/run HTTP/1.1" 200 771 "http://localhost:3000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3.1 Safari/605.1.15"
livekit-1 | 2025-08-20T17:45:21.143Z INFO livekit agent/client.go:173 failed to send job request {"error": "no servers available (received 1 responses)", "namespace": "", "jobType": "JT_ROOM", "agentName": "gabber-engine"}
livekit-1 | 2025-08-20T17:45:21.941Z INFO livekit agent/client.go:173 failed to send job request {"error": "no response from servers", "namespace": "", "jobType": "JT_ROOM", "agentName": ""}

(not sure why it thinks I'm on an Intel Mac!)

[keepingitneil]

Intel mac I assume is because you are running in docker and perhaps you installed docker on the wrong arch (?). The hello world is meant to be used as follows:

• Click Run
• Turn on video and/or audio
• See and/or hear yourself

Wrapping up a couple of small things then I'll go through docker on mac and test thoroughly to reproduce. I do most of my testing with the make all workflow because it has filewatchers and is faster to iterate. I did most of the docker testing on my linux machine so it's possible that the differences on mac (i.e. maybe host.docker.internal behavior) are unaccounted for

[scalar27]

OK, I guess I figured it out! I switched to another browser and the errors went away. Is it known not to work in Safari? I got it to work with ARC... Playing with it now.