Related Posts
Conversation Starter
Conversation Starter
New to Fishbowl?
Download the Fishbowl app to
unlock all discussions on Fishbowl.
unlock all discussions on Fishbowl.
That was just a preview…
Sign Up to see all discussions
Sign Up to see all discussions
- Discover what it’s like to work at companies from real professionals
- Get candid advice from people in your field in a safe space
- Chat and network with other professionals in your field
Sign up in seconds to unlock all discussions on Fishbowl.
"" is an unusual email domain. If correct, please proceed to the next step
Messaging rates may apply
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
New to Fishbowl?
Download the Fishbowl app to unlock all discussions on Fishbowl.
Share
Embed this post
Copy and paste embed code on your site
Preview
Download the
Fishbowl app
Scan your QR code to download
Fishbowl app on your mobile
Sign up for free to view this conversation on Fishbowl
Sign up for free to continue using Fishbowl
Messaging rates may apply
It's all just really really good pattern matching.
You can't really trust it to get any of this stuff right consistently because it still has no idea what you're saying or what it's saying back.
We are very good at seeing emergent performance when it isn't really there.
A dedicated, narrow model is still going to be a better solution for some use cases.
Me? Nah not formally, but I've done some interesting personal projects with things like parsing out 10-Ks and doing sentiment analysis and categorization on the risk factors section and using that as input to time series models. It ain't perfect, but it's surprisingly good for something that was not directly trained on summarization, sentiment, categorization etc ML oriented tasks
Much of the emergent ability comes from the generalizability of the training process, which includes both the supervised fine tuning, reward model, and RLHF process. The foundational language model itself already had zero-shot ability due to its understanding of text, but it wasn’t well focused. That’s why we can get pretty good performance for tasks on unseen data.
But once you add in the other training steps, the model becomes optimized for answering prompts in a way that’s sensible for people, since that’s exactly what it was trained on. The objective task in this case is broad enough to encompass things like summarization, sentiment, NER, etc. And with a large enough model, it does a pretty good job of these things.
Given the same model size /data size / computation, task specific models will almost always perform better. And even with much smaller models/data size/computation, task specific modes often still perform better.
Thanks for the insight! Seems like so many "use cases" that consulting firms are preaching about would be much better achieved with task specific models, do you get the same feeling? Do you think there's the potential for broad use here or will task specific models still be the right way to go in the near term?
Try out https://app.withmaterial.com/ Looking forward to your feedback!