Fine-Tuning Dataset Checker

Find problems in your chat JSONL training data before you pay for a fine-tuning run.

  • JSONL validation
  • Token counts
  • Runs in your browser

Did this save you some time?

aiml.site costs $20 a year to run and every tool stays free. If it helped, you can buy me a dosa.

Loading…

Need a custom web app?

I built every tool here and take freelance work: fast, focused tools for your business.

Email hello@aiml.site

How to use Fine-Tuning Dataset Checker

  1. Open your .jsonl file or paste the lines.

  2. Read the summary: examples, errors, warnings and token counts.

  3. Fix the listed lines, then run the check again before uploading.

How it works

Each line is parsed as JSON on its own. The checker then confirms there is a "messages" array, that every message has a known role (system, developer, user, assistant or tool) and non-empty content, that each example has at least one assistant reply, and that assistant weight values are 0 or 1. It also flags system messages that are not first and exact duplicate examples. Tokens are counted with the o200k_base tokenizer from js-tiktoken. All of this runs in your browser.

Why use it

  • Catches errors before a paid training run
  • Line-by-line issue list
  • Token totals for cost estimates
  • Duplicate detection
  • Your data never leaves your device

Questions

What format does it check?

The chat format used by OpenAI and many open-source trainers: one JSON object per line with a "messages" array of {role, content} objects.

Which problems does it find?

Invalid JSON lines, missing or empty messages arrays, unknown roles, empty content, examples with no assistant reply, bad weight values, misplaced system messages and exact duplicates.

How are tokens counted?

With the o200k_base tokenizer from js-tiktoken, which matches recent OpenAI models. Other providers use different tokenizers, so treat the numbers as close estimates for them. Message formatting adds a few tokens per message on top.

Is my data uploaded?

No. The file is read and checked in your browser. Nothing is sent to a server.

Does it work for Llama or Mistral fine-tuning?

Yes, as long as your trainer uses the same messages format. Check your trainer's docs for its exact rules.

How many examples do I need?

Most providers require at least 10. In practice, start with 50 to 100 high-quality examples and add more if results are weak.