Configure Data Settings (For Preference Tuning)

• While configuring the dataset metadata:

  • Data Type: Select “text”
  • Task: Choose “llm-preference-tuning”

These values ensure the dataset is correctly categorized for preference-based learning.

• Prepare Your File (CSV Format)

  
  

  Your CSV file must include the following columns:

  • prompt (Required): The input instruction or query given to the model.
  • chosen (Required): The preferred response.
  • rejected (Required): The less preferred or incorrect response.

💡 Column names must be lowercase. Avoid empty fields in required columns.

• Upload and Finalize

  • Upload the file via drag-and-drop or Select File option.
  • Ensure the file uploads successfully, then click “Finish”.
  • Wait for the success confirmation.
  • Locate your dataset, verify metadata, and Save.
  • Confirm that the status is marked “annotated”.
  
  

  Once completed, your preference tuning dataset is ready for training your LLM to better distinguish between high- and low-quality responses.