31 lines
1.4 KiB
Markdown
31 lines
1.4 KiB
Markdown
# DSPy OpenTOM
|
|
|
|
This repo contains scripts for optimizing DSPy modules for the OpenTOM Benchmark. We support Chain of Thought and a method we thought might work where we generate a "thought" about the context to aid in answering the question (spoiler -- it didn't work better than just `BootstrapFewShotWithRandomSearch`).
|
|
|
|
CLI Usage:
|
|
```
|
|
usage: main.py [-h] [--student STUDENT] [--teacher TEACHER] [--train_size TRAIN_SIZE] [--download_dataset DOWNLOAD_DATASET]
|
|
[--question_types [QUESTION_TYPES ...]]
|
|
experiment_title dspy_method dspy_optimizer
|
|
|
|
Run DSPY method.
|
|
|
|
positional arguments:
|
|
experiment_title Title of new experiment
|
|
dspy_method The DSPY method to run
|
|
dspy_optimizer The DSPY optimizer to use
|
|
|
|
options:
|
|
-h, --help show this help message and exit
|
|
--student STUDENT The LLM to optimize prompts for
|
|
--teacher TEACHER Teacher LLM for optimizing prompts. Defaults to Student LLM
|
|
--train_size TRAIN_SIZE
|
|
Number of training examples to use for optimization
|
|
--download_dataset DOWNLOAD_DATASET
|
|
Download dataset
|
|
--question_types [QUESTION_TYPES ...]
|
|
Question types. Defaults to all
|
|
```
|
|
|
|
Come chat with us in our [discord](https://discorg.gg/plasticlabs) or in the [DSPy thread](https://discord.com/channels/1161519468141355160/1214629969318252574)
|