Code

[DRAFT] prompt migration engine#808

Code

Draft

heyjustinai wants to merge 38 commits intomainmeta-llama/llama-cookbook:mainfrom

prompt-migrationmeta-llama/llama-cookbook:prompt-migrationCopy head branch name to clipboard

Commits

Commits on Jan 15, 2025

• added evaluator and formatter and main

  heyjustinai

  committed
  2570d16

  Copy full SHA for 2570d16

• add usage guide and init

  heyjustinai

  committed
  08e41d0

  Copy full SHA for 08e41d0

• add engine and eval dataset

  heyjustinai

  committed
  a3e96e4

  Copy full SHA for a3e96e4

• add .env settings and configure yml

  heyjustinai

  committed
  096249b

  Copy full SHA for 096249b

• placeholder readme

  heyjustinai

  committed
  263b8b5

  Copy full SHA for 263b8b5

• adding eval dataset

  heyjustinai

  committed
  43a2cbc

  Copy full SHA for 43a2cbc

• change eval dataset, include more robust judging, improved main

  heyjustinai

  committed
  b85811d

  Copy full SHA for b85811d

• minor changes in eval, deleted formatter

  heyjustinai

  committed
  90d16cd

  Copy full SHA for 90d16cd

• update dir

  heyjustinai

  committed
  4d75fe9

  Copy full SHA for 4d75fe9

Commits on Jan 16, 2025

• updated prompt migration to use benchmark and also mipro, added meta implementation

  heyjustinai

  committed
  e52e1d1

  Copy full SHA for e52e1d1

• update harness notebook

  heyjustinai

  committed
  1e4c6d2

  Copy full SHA for 1e4c6d2

• update harness notebook

  heyjustinai

  committed
  62b5367

  Copy full SHA for 62b5367

• beef up readme

  heyjustinai

  committed
  5730a84

  Copy full SHA for 5730a84

Commits on Jan 17, 2025

• added updated llama-mmlu-pro and added human-eva

  heyjustinai

  committed
  314b6a8

  Copy full SHA for 314b6a8

• harness runcode

  heyjustinai

  committed
  2776a35

  Copy full SHA for 2776a35

Commits on Jan 21, 2025

• updated readme

  heyjustinai

  committed
  0bec41f

  Copy full SHA for 0bec41f

• change gpu parallel size docs

  heyjustinai

  committed
  03f2b8e

  Copy full SHA for 03f2b8e

Commits on Jan 22, 2025

• attempt to fix json output format in eval

  heyjustinai

  committed
  becbe77

  Copy full SHA for becbe77

• <Replace this line with a title. Use 1 line only, 67 chars or less>

  Show description for a6f448f
  heyjustinai

  committed
  a6f448f

  Copy full SHA for a6f448f

• revert to previous changes

  heyjustinai

  committed
  4fd5f29

  Copy full SHA for 4fd5f29

• batching and parallelization, ran on baseline and lite

  heyjustinai

  committed
  eea9661

  Copy full SHA for eea9661

Commits on Jan 24, 2025

• added inspect and modified harness

  heyjustinai

  committed
  9ffb292

  Copy full SHA for 9ffb292

Commits on Jan 27, 2025

• updated env file

  heyjustinai

  committed
  8d3a047

  Copy full SHA for 8d3a047

Commits on Jan 29, 2025

• added fix split, gitignore and download mmlu script

  heyjustinai

  committed
  e19b9e9

  Copy full SHA for e19b9e9

• update mmlu pro

  heyjustinai

  committed
  21e04c2

  Copy full SHA for 21e04c2

• setup meta-eval for benchmark, ray error

  heyjustinai

  committed
  dc406b4

  Copy full SHA for dc406b4

Commits on Jan 30, 2025

• Merge pull request #2 from pia-papanna/tools-refactory-chester

  Show description for 07b191b
  

  WuhanMonkey

  authored and

  heyjustinai

  committed
  07b191b

  Copy full SHA for 07b191b

• running mmlu pro with meta eval - fixed error

  heyjustinai

  committed
  f8a6c7d

  Copy full SHA for f8a6c7d

• update utils

  heyjustinai

  committed
  caeddcc

  Copy full SHA for caeddcc

• updated mmlu meta-eval for prompt migration

  heyjustinai

  committed
  479b1fb

  Copy full SHA for 479b1fb

• update gitignore, added mmlu 0shot and ran a bunch of test

  heyjustinai

  committed
  e1d64ca

  Copy full SHA for e1d64ca

• Stop tracking files in eval_results/meta-llama__Llama-3.3-70B-Instruct

  heyjustinai

  committed
  d214437

  Copy full SHA for d214437

• updated gitignore

  heyjustinai

  committed
  d4638ba

  Copy full SHA for d4638ba

• update readme

  heyjustinai

  committed
  7a014b3

  Copy full SHA for 7a014b3

• made changes to utils

  heyjustinai

  committed
  52c5a76

  Copy full SHA for 52c5a76

Commits on Feb 3, 2025

• updated mmlu and harness

  heyjustinai

  committed
  423231e

  Copy full SHA for 423231e

Commits on Feb 4, 2025

• code handoff

  heyjustinai

  committed
  3174e5b

  Copy full SHA for 3174e5b

Commits on Feb 27, 2025

• merge utils, added configs and sample notebook to run optimizer

  heyjustinai

  committed
  4768a41

  Copy full SHA for 4768a41