Great progress but pretty unusable in the real world still
It goes off the rails completely with relative ease, but the duplex ability is really impressive, this is the future direction of speech-to-speech for sure.
Are there any colab notebooks or a hosted version to try?
The entire 7B model is like 14 GB and was only trained on ~2000 hours of conversation. If that were in text form, it would only take up ~30 MB.
For comparison, GPT-2 was trained on around 40 GB of text, and GPT-3 was trained on nearly 500 GB of text. This is just to say that a model which performs as well as you'd probably like is going to require much more training and won't fit on a GPU which you can install in your computer.
Can you even imagine how many will lose job in support lines because of this, esp in India....i suspect something of that already used on banking support lines, which today first block your transaction, next call you to just find out its you (the dialogue there very small and short) and this is made without any laws, all Ai examples in real life i've heard made in absence of legal field (hiding that LLM is used is kinda illegal because LLM should never lie itself, if human lie of using Ai covertly-thats fraud by default, depends on outcome). Implementation of Ai in all contact and support departments (like Google) legally looks like isolation from any responsibility, any legal request becoming very lengthy and hard process, almost impossible without real lawyers and courts. Ai have the power today to block finances in many countries banks without any human approvals.
##My Hardware## Intel Xeon E5-2699v4 LGA2011-3 22 cores 44 threads (2016) $110 # Gigabyte C612 chipset 12 RAM slots VGA motherboard year 2016 $150 # Samsung-Hynix ECC RAM 12x64Gb=768Gb ~$900 # VGA monitor # IKEA chair # Run: Trillions Deepseeks, Kimis in Q5-Q6, 400-500billions in BF16