"The device does not support fp16."
#1
by
utopiah
- opened
for Chromium Version 126.0.6452.0 (Developer Build) (64-bit) on Linux with CUDA enabled (if that matters)
Linux hasn't had great support for fp16 unfortunately... We could do a check beforehand, but if it fails, you would need to load the 1.76GB fp32 vision encoder and 420MB fp32 embedding layer (as opposed to 880MB and 210MB, respectively).
Thanks for the clarification. I personally wouldn't mind as on desktop as I would expect to have a proper connection.