ToTs Trees-Of-Thought

TODO: Watch/follow with this video regarding looping as part of ToTs https://www.youtube.com/watch?v=j320H2LFx-U

Visualization

Figure 1: Schematic illustrating various approaches to problem solving with LLMs. Each rectangle box represents a thought, which is a coherent language sequence that serves as an intermediate step toward problem solving.

Process

Step 1: Brain Storming Phase

From ToTs Step1: Brain Storming Phase

Go to text →

Purpose: Generate diverse range of solutions to a problem.

Get the LLM to put out high level ideas.

Get LLM to provide reasoning, preferably stating the factors to consider.

The first phase of the Tree of Thought process involves brainstorming diverse potential solutions to a given problem. In this stage, you can ask your AI model to generate three or more options while considering various factors. allabtai.com

Example from All About AI

> Prompt: I have a problem related to [describe your problem area]. Could you brainstorm three distinct solutions? Please consider a variety of factors such as [Your perfect factors] - [ref](https://www.allabtai.com/the-tree-of-thoughts-prompt-template/)

Templated Example without Markdown

You are: <INSERT_PREPPING_ROLE_DESCRIPTION_HERE>

I have the following problem

My_Problem=<INSERT_YOUR_PROBLEM_HERE>.

My_Background_In_Relation_To_This_Problem=<INSERT_YOUR_BACKGROUND_HERE_OR_DELETE_THIS_LINE>

Could you brainstorm and present multiple distinct solutions? Please explain your reasoning for including each option. Please consider a variety of factors including but not limited to: [<INSERT_YOUR_FACTORS_HERE>]

Also give each option a sequenced number and a random id, so that we can reference this id later. An example of {RandomId} is as3133bn.

Next step: ToTs Step2: Evaluation Phase

Step 2: Evaluation Phase

From ToTs Step2: Evaluation Phase

Go to text →

Purpose:

Assess feasibility and potential success of each option.
Evaluate pros and cons.
Confidence levels and probabilities.

The second phase is where the AI model objectively assesses each option’s potential success by evaluating their pros and cons, initial effort, implementation difficulty, potential challenges, and expected outcomes. The AI assigns a probability of success and a confidence level for each option based on these factors. allabtai.com

All About AI Example

Prompt: For each of the three proposed solutions, evaluate their potential. Consider their pros and cons, initial effort needed, implementation difficulty, potential challenges, and the expected outcomes. Assign a probability of success and a confidence level to each option based on these factors - ref

A more generalized prompt

Prompt:

For each of the proposed solutions, evaluate their potential.

Consider their pros and cons. Think of dimension: [{INCLUDE_DIMENSIONS}, potential-challenges], but do limit to only these dimensions.

Assign a probability of success and a confidence level to each option based on these factors.

Previous Step: ToTs Step1: Brain Storming Phase
Next Step: ToTs step3: Expansion Phase

Step 3: Expansion Phase

From ToTs step3: Expansion Phase

Go to text →

ToTs | allabtai video | Timestamped

The third phase involves delving deeper into each idea, refining it, and imagining its implications in real-world contexts. The AI model generates potential scenarios, strategies for implementation, necessary partnerships or resources, and possible ways to overcome obstacles. allabtai.com

allabtai.com Prompt Example

Prompt: For each solution, deepen the thought process. Generate potential scenarios, strategies for implementation, any necessary partnerships or resources, and how potential obstacles might be overcome. Also, consider any potential unexpected outcomes and how they might be handled. - ref

Previous Step: ToTs Step2: Evaluation Phase
Next step: ToTs Step-4: Decision Phase

Step 4: Decision Phase

From ToTs Step-4: Decision Phase

Go to text →

During the final phase, the AI model ranks each solution based on the evaluations and scenarios generated. It provides justifications for its rankings and offers any final thoughts or considerations for each solution. - ref

allabtai.com prompt example

Prompt: Based on the evaluations and scenarios, rank the solutions in order of promise. Provide a justification for each ranking and offer any final thoughts or considerations for each solution - ref

Previous Step: ToTs step3: Expansion Phase

Claims

Claims: 4% -> 74% success rate for Game of 24 while using ChatGPT-4, 1750% improvement

Our experiments show that ToT significantly enhances language models' problem-solving abilities on three novel tasks requiring non-trivial planning or search: Game of 24, Creative Writing, and Mini Crosswords. For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%. - paper