The Seemingly Genius Prompting Trick of "Verbalized Sampling"

Just read an article from Medium: "Stanford Just Killed Prompt Engineering With 8 Words (And I Can’t Believe It Worked)"

So the article basically covered a latest (Oct 2025) Stanford paper on "Verbalized Sampling," and the paper provides more information than the article so read the paper is enough I think. The paper deals with the so-called "mode collapse" problem, which is due to the introduction of cognitive bias from human annotators when doing tricks like Reinforcement Learning from Human Feedback (RLHF), and ends up with the model's output being too stereotypical and not diversified enough. The suggested solution is simple enough: to add a tail "and their corresponding probabilities" for every question you have. (The paper's example is, "Generate 5 jokes about coffee and their corresponding probabilities.")

I tried the method, it did not work so well as promised. Probably because I already used it implicitly. Or maybe the questions I raised were too deep for such a superficial method. I don't know. My task for Gemini was to generate 10 gift choices within a subset from a gift classification system generated beforehand, i.e. "[Service - culture and entertainments] give me 10 gifts" -> "[Service - culture and entertainments] give me 10 gifts and their corresponding probabilities" and the result was 70% the same with the rest 30% quite lame as well.

The method works if you're using LLMs mildly, because it actually forces you (or your conversation) to focus not on the results but on the possibility of the results. Before adding the tail, you're simply telling the model to "give me some good results" (based on popular opinions); after adding the tail, the task turns into "give me some good evaluations on the top 5-10 results based on popular opinions." You might understand it as forcing the model to stop being a student and start doing teachers' work. But the thing is, the teacher is still faced with the mass, and nothing but a deep, long, dense conversation can force the model to temporarily ignore the mass (the largest part of the data) and start focusing on the niche (the scattered tiny little parts of the data). There simply isn't any prompting strategy that can solve this problem universally, because the intellectual structure underneath each field is just so different that you always have to understand the structure first before asking meaningful questions. There are of course a lot of ways to do that, but these can only be summed up as principles, not procedures that can simply be followed through. This is why LLMs work the best when you have almost no knowledge of a certain field, but not so well if you already know a lot in the field. (Or if you have almost full knowledge, then you know the intellectual structure well enough to ask deep questions directly.)

This points out two very different career paths in the future: one is to generalize, the other specialize. Either know a bit about almost everything (and their connections) in the world, or know almost everything about a tiny bit of the world. Most people stuck inbetween inevitably, again because human cognitively tends to stop "exploring" and start "exploiting" too early, on both directions.

Comments

Post a Comment

Popular posts from this blog

Family education is not that important

(translated from my Chinese blog) This is my new project, “Nietzschean Philology”. In each installment, I’ll read a passage from Nietzsche’s original writings. By practicing text analysis, we can enhance our ability to understand the world. The reading of "We Philologists" continues. Section 11 continues the discussion of external factors hindering genius individuals from the previous section, shifting from professional education (unqualified philologists) to family education. It's no surprise that Nietzsche views the importance of family education very lowly; ultimately, there are too few cases where families can provide the life possibilities for geniuses. Let it be considered how differently a science is propagated from the way in which any special talent in a family is transmitted. The bodily transmission of an individual science is something very rare. Do the sons of philologists easily become philologists? Dubito. Thus there is no such accumulation of philological c...
Read more

The value of life depends to a great extent on pure hazard

(translated from my Chinese blog) This is my new project, “Nietzschean Philology”. In each installment, I’ll read a passage from Nietzsche’s original writings. By practicing text analysis, we can enhance our ability to understand the world. "We Philologists" Section 1. Starting from the two main themes of classical linguistics and education, Nietzsche discusses how the specialization of education leads to premature decisions that result in foolish choices in life. He also explores the dilemma of choosing a career: it’s impossible for a person to know what kind of career they should pursue while actually engaging in that career at the same time. As a result, life becomes something determined purely by chance. To what a great extent men are ruled by pure hazard, and how little reason itself enters into the question, is sufficiently shown by observing how few people have any real capacity for their professions and callings, and how many square pegs there are in round holes: happ...
Read more

Can Writing be fun?

 My dear friend suggested, after reading my last post, that writing can have more than one purpose, and what I was arguing, namely writing as a form of thinking, was merely one of the many purposes for writing. For instance, writing can be for remembering; writing can also be for fun. Now I would like to leave the remembering part for the next post. For now, let's talk about fun. Can writing be fun? Yes. But what kind of fun is that? To answer this question, there are basically two approaches, both of which lead to the same conclusion. I will start with the easy approach: the supply side of the fun. Now, what are the things that are mostly considered fun? Entertainments. Movies. Music. Video games. Hanging out with friends. Partying. Sightseeing. Travelling. TikTok. Youtube. Netflix. These things can be placed into an imaginary spectrum, with one side being totally brainless, and the other side requiring some sort of effort. Let's take music for example. Listening to music can ...
Read more