ToTs Trees-Of-Thought
TODO: Watch/follow with this video regarding looping as part of ToTs https://www.youtube.com/watch?v=j320H2LFx-U
Visualization
Visualization
Figure 1: Schematic illustrating various approaches to problem solving with LLMs. Each rectangle
box represents a thought, which is a coherent language sequence that serves as an intermediate
step toward problem solving.
Process
Step 1: Brain Storming Phase
Purpose: Generate diverse range of solutions to a problem.
Get the LLM to put out high level ideas.
Get LLM to provide reasoning, preferably stating the factors to consider.
The first phase of the Tree of Thought process involves brainstorming diverse potential solutions to a given problem. In this stage, you can ask your AI model to generate three or more options while considering various factors. allabtai.com
Example from All About AI
> Prompt: I have a problem related to [describe your problem area]. Could you brainstorm three distinct solutions? Please consider a variety of factors such as [Your perfect factors] - [ref](https://www.allabtai.com/the-tree-of-thoughts-prompt-template/)Templated Example without Markdown
You are: <INSERT_PREPPING_ROLE_DESCRIPTION_HERE>
I have the following problem
My_Problem=<INSERT_YOUR_PROBLEM_HERE>.
My_Background_In_Relation_To_This_Problem=<INSERT_YOUR_BACKGROUND_HERE_OR_DELETE_THIS_LINE>
Could you brainstorm and present multiple distinct solutions? Please explain your reasoning for including each option. Please consider a variety of factors including but not limited to: [<INSERT_YOUR_FACTORS_HERE>]
Also give each option a sequenced number and a random id, so that we can reference this id later. An example of {RandomId} is as3133bn
.
Navigation
Next step: ToTs Step2: Evaluation Phase
Step 2: Evaluation Phase
Purpose:
- Assess feasibility and potential success of each option.
- Evaluate pros and cons.
- Confidence levels and probabilities.
The second phase is where the AI model objectively assesses each option’s potential success by evaluating their pros and cons, initial effort, implementation difficulty, potential challenges, and expected outcomes. The AI assigns a probability of success and a confidence level for each option based on these factors. allabtai.com
All About AI Example
Prompt: For each of the three proposed solutions, evaluate their potential. Consider their pros and cons, initial effort needed, implementation difficulty, potential challenges, and the expected outcomes. Assign a probability of success and a confidence level to each option based on these factors - ref
A more generalized prompt
Prompt:
For each of the proposed solutions, evaluate their potential.
Consider their pros and cons. Think of dimension: [{INCLUDE_DIMENSIONS}, potential-challenges], but do limit to only these dimensions.
Assign a probability of success and a confidence level to each option based on these factors.
Navigation
- Previous Step: ToTs Step1: Brain Storming Phase
- Next Step: ToTs step3: Expansion Phase
Step 3: Expansion Phase
ToTs | allabtai video | Timestamped
The third phase involves delving deeper into each idea, refining it, and imagining its implications in real-world contexts. The AI model generates potential scenarios, strategies for implementation, necessary partnerships or resources, and possible ways to overcome obstacles. allabtai.com
allabtai.com Prompt Example
Prompt: For each solution, deepen the thought process. Generate potential scenarios, strategies for implementation, any necessary partnerships or resources, and how potential obstacles might be overcome. Also, consider any potential unexpected outcomes and how they might be handled. - ref
Navigation
- Previous Step: ToTs Step2: Evaluation Phase
- Next step: ToTs Step-4: Decision Phase
Step 4: Decision Phase
During the final phase, the AI model ranks each solution based on the evaluations and scenarios generated. It provides justifications for its rankings and offers any final thoughts or considerations for each solution. - ref
allabtai.com prompt example
Prompt: Based on the evaluations and scenarios, rank the solutions in order of promise. Provide a justification for each ranking and offer any final thoughts or considerations for each solution - ref
Navigation
- Previous Step: ToTs step3: Expansion Phase
Claims
Claims: 4% -> 74% success rate for Game of 24 while using ChatGPT-4, 1750% improvement
Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. - paper
Links
Easier to Digest Links
All About AI
Medium
Research Links
Children
- Assign Ids to Options
- ToTs Step-4: Decision Phase
- ToTs Step1: Brain Storming Phase
- ToTs Step2: Evaluation Phase
- ToTs step3: Expansion Phase
- be-careful-with-using-markdown-table-formatting
Backlinks