Conduct hyperparameter search over generation parameters of large language models (LLMs). This tool is designed for ML practitioners looking to optimize their sampling strategies to improve model performance. Simply provide a model, dataset, and performance metric, llmsearch handles the rest.

QuickStart

• llama-3-8b Example - A quickstart notebook which shows the basic functionality of llmsearch. This notebook will help you understand how to quickly set up and run hyperparameter searches.

End-to-End Model Examples

1. GSM8K Example - Shows a GridSearchCV ran on the GSM8K Dataset using the TheBloke/CapybaraHermes-2.5-Mistral-7B-AWQ model.
2. Samsum Example - Shows a GridSearchCV ran on the samsum Dataset using a finetuned(on the same dataset) version of cognitivecomputations/dolphin-2.2.1-mistral-7b.

Refer README further for more details on how to use llmsearch.

API Reference Documentation

Sub-modules

llmsearch.patches

Module which is used to monkey patch transformers library.

llmsearch.scripts

Useful utility scripts such as Multi Token Stopping Criteria.

llmsearch.tuner

Core Tuner Class that adapts a PyTorch model to function as a scikit-learn estimator.

llmsearch.utils

Contains useful utilities that help with the functioning of llmsearch.