Samsung Electronics has unveiled its own generative AI model, Samsung Gauss, which is to be incorporated into its Galaxy S24 series in the first half of 2024, as part of the Korean tech giant's bold move to join the fiercely competitive generative AI race in the global IT industry.
So it will be locally hosted on the phone? I seriously doubt it will be very useful in offline only mode. Even relativity small language models (7B or 13B) struggle on even desktop pc’s if you don’t have a high end graphics card with 12+ GB of vram. Analyzing can be relatively fast, but generating will be terribly slow, especially images.
Edit: So after some reading, the snapdragon gen 3 has some impressive specs, but can someone explain how a phone can generate fast ai content while a PC needs let’s say 24GB’s of vram? I get the phone has an ai-specialized chip, but you still need to load the model into memory.
They said it would be available on their upcoming flagship product, sure, that’s marketing. To me, this does not imply that the heavy-lifting is on the phone’s hardware.
(And maybe a special-purpose AI, which requires significantly smaller models, can run on the phone, which is a high-end mobile computer.)
So it will be locally hosted on the phone? I seriously doubt it will be very useful in offline only mode. Even relativity small language models (7B or 13B) struggle on even desktop pc’s if you don’t have a high end graphics card with 12+ GB of vram. Analyzing can be relatively fast, but generating will be terribly slow, especially images.
Edit: So after some reading, the snapdragon gen 3 has some impressive specs, but can someone explain how a phone can generate fast ai content while a PC needs let’s say 24GB’s of vram? I get the phone has an ai-specialized chip, but you still need to load the model into memory.
Did they say, it would run locally?
They said it would be available on their upcoming flagship product, sure, that’s marketing. To me, this does not imply that the heavy-lifting is on the phone’s hardware.
(And maybe a special-purpose AI, which requires significantly smaller models, can run on the phone, which is a high-end mobile computer.)
If you don’t need an incredible amount of creativity, quantization can get it to work with relatively little RAM.