Data annotation with a crowd annotation platform

You can use a third-party crowd annotation platform to improve the quality of your training data. Through Watson Studio, you can integrate a crowd annotation platform and create an annotation job. Your data for that job is sent to the annotation platform, where external crowd contributors annotate your data. Contributors are tested to ensure eligibility and monitored for trustworthiness. You can choose how many contributor responses you receive per row. You receive an annotated data set as a new data asset. You can see a quality report at the conclusion of your annotation job.

These crowd annotation platforms are integrated with Watson Studio:

  • The Figure Eight platform. Figure Eight integration is currently in beta. You can create annotation jobs for sentiment analysis on CSV files. You can also annotate and label image files.
  • The DefinedCrowd platform. DefinedCrowd integration is currently in beta. You can create annotation jobs for sentiment analysis on CSV files and you can annotate and label image files. You need a DefinedCrowd enterprise account.

Watson Studio plan and role requirements:

  • Crowd annotation is available with only some Watson Studio plans.
  • To configure crowd annotation integration, you must have the Admin role in the project.
  • To create annotation jobs, you must have the Admin or Editor role in the project.

Data privacy

Only the data that you specify in the annotation job are sent to the annotation platform. When you submit the annotation job, your data is subject to the privacy policy of the crown annotation platform.

Learn more