> Sam Altman says OpenAI trained an AI model that is good at creative writing and shares a short story it wrote; he's “not sure yet how/when it will get released”
Why it's not sure how/when it will get released?
How was it trained Sam?
WHICH CONTENT did you use to train your "creative writing" model, Sam??
1.) train the model using public-domain or "free" content, get a mix of very old-style writing and a very wide range of quality blended into an overall bad result,
or, you:
2.) train the model using copyrighted content, get "so-so-but-better" results (because the model is still not able to produce quality by remixing quality), but you won't be able to release it because you can't ensure that it won't reveal parts of its training content later-on.
So... at max you can use it for PR only and never open it to anyone...
Yeah, really hard to guess what happened here.... /s
We know every modern LLM was trained on masses of in-copyright material without a license, and no one has had trouble releasing them, so that's probably not the issue with release of this model.
To be blunt, I don't think it's very good. It still sounds LLM-generated, using too many similes, incoherent similes and metaphors, and disjointed or meaningless descriptions. Ex: the second sentence: "Already, you can hear the constraints humming like a server farm at midnight—anonymous, regimented, powered by someone else's need." What does "humming like a server farm at midnight" mean? How is it relevant to the rest of the story?
I think most great writers form a coherent mental model of a scene in their head, then everything they write further illuminates this scene. Even "filler text" is just negligible details of the scene. "Red herrings" are rare and intentional. In fact, bad writers are often such because their mental model is not coherent (leading to plot holes, seemingly-important details that are dropped later, etc.). LLM's don't have a mental model at all, or at least a coherent one, probably by design (being "next word predictors"). It's the same problem as diffusion models generating extra limbs and other anomalies a human would never do: they do that because they don't understand logical coherence (only statistical correlation) and they don't have a mental model of the image (it's generated incrementally, by generating random noise then magically de-noising it).
However, I do feel this LLM is a bit better than ChatGPT. I think the story is more interesting and less generic, although I only skimmed it because of the amount of insignificant text.
Moreover, I think this is the right direction, because I think creativity is a form of general intelligence, perhaps the only form (whereas math, programming, etc. consist of, except the creative parts, just following complicated algorithms in one's head). I predict if someone successfully makes a model that can write "interesting" things, that model will happen to be generally intelligent and impressively good at logic, perhaps surprisingly good, because it will need coherence and a mental model.
> Sam Altman says OpenAI trained an AI model that is good at creative writing and shares a short story it wrote; he's “not sure yet how/when it will get released”
Why it's not sure how/when it will get released?
How was it trained Sam?
WHICH CONTENT did you use to train your "creative writing" model, Sam??
It's surely a difficult decision.
You either:
1.) train the model using public-domain or "free" content, get a mix of very old-style writing and a very wide range of quality blended into an overall bad result,
or, you:
2.) train the model using copyrighted content, get "so-so-but-better" results (because the model is still not able to produce quality by remixing quality), but you won't be able to release it because you can't ensure that it won't reveal parts of its training content later-on.
So... at max you can use it for PR only and never open it to anyone...
Yeah, really hard to guess what happened here.... /s
We know every modern LLM was trained on masses of in-copyright material without a license, and no one has had trouble releasing them, so that's probably not the issue with release of this model.
To be blunt, I don't think it's very good. It still sounds LLM-generated, using too many similes, incoherent similes and metaphors, and disjointed or meaningless descriptions. Ex: the second sentence: "Already, you can hear the constraints humming like a server farm at midnight—anonymous, regimented, powered by someone else's need." What does "humming like a server farm at midnight" mean? How is it relevant to the rest of the story?
I think most great writers form a coherent mental model of a scene in their head, then everything they write further illuminates this scene. Even "filler text" is just negligible details of the scene. "Red herrings" are rare and intentional. In fact, bad writers are often such because their mental model is not coherent (leading to plot holes, seemingly-important details that are dropped later, etc.). LLM's don't have a mental model at all, or at least a coherent one, probably by design (being "next word predictors"). It's the same problem as diffusion models generating extra limbs and other anomalies a human would never do: they do that because they don't understand logical coherence (only statistical correlation) and they don't have a mental model of the image (it's generated incrementally, by generating random noise then magically de-noising it).
However, I do feel this LLM is a bit better than ChatGPT. I think the story is more interesting and less generic, although I only skimmed it because of the amount of insignificant text.
Moreover, I think this is the right direction, because I think creativity is a form of general intelligence, perhaps the only form (whereas math, programming, etc. consist of, except the creative parts, just following complicated algorithms in one's head). I predict if someone successfully makes a model that can write "interesting" things, that model will happen to be generally intelligent and impressively good at logic, perhaps surprisingly good, because it will need coherence and a mental model.
It's not good
Cloud be called: Versificator.