iMerit vs. CloudFactory: Which to Choose for Your ML Project

In the rapidly growing field of data labeling for machine learning, iMerit and CloudFactory are among the leading service providers. iMerit boasts a workforce of over 5,500 skilled annotators, delivering high-quality services to tech giants like Microsoft and eBay. CloudFactory, on the other hand, leverages a global talent pool to provide scalable solutions, supporting companies such as Hummingbird Technologies and Drive.ai​​.

To save you time and hours of research, we’ve compared their offerings to help you decide which data labeling company best fits your ML project.

iMerit vs. CloudFactory: Company Profiles

iMerit Company

Founded in 2012 by Dipak Basu, iMerit counts over 5000 employees. It provides full end-to-end services for data annotation in a wide array of industries. The company is headquartered in San Jose, California, but also has offices in New Orleans, LA, Kolkata, and Bengaluru, India.

CloudFactory Company

CloudFactory provides human-in-the-loop (HITL) data labeling solutions, utilizing a global, on-demand workforce and AI technology. Trusted by over 700 AI companies, the company provides a large talent pool of more than 7,000 data analysts. Founded by Mark Sears in 2010, CloudFactory has offices in the UK, US, Nepal, and Kenya, with Kevin Johnston as the new CEO in 2024.

Services and Products

iMerit Services and Products

The iMerit company provides data annotation services for generative AI, computer vision, natural language processing (NLP), as well as content services. Their main areas of focus include:

• Reinforcement learning from human feedback (RLHF) for LLMS and LVMS

• Image annotation

• Video annotation

• Text annotation

• Audio transcription

• LiDAR annotation

• Sentiment analysis

• Content moderation

• Product categorization

• Image segmentation

Their annotators work with different languages. The majority of them are English-speaking and are based in India. Yet, there are also teams in the US, Bhutan, Germany, and Latin America, who can work on your multilingual ML projects.

CloudFactory Services and Products

CloudFactory delivers human-in-the-loop AI solutions, from data curation and annotation, to QA and model optimization. Their core services include:

Data Labeling:

• Accelerated Annotation

• Workforce Plus (workforce + tech)

• Vision AI Managed Workforce

• NLP

• Data Processing

Human-in-the-Loop Automation:

• Managed Workforce

CloudFactory’s data annotation services are mostly focused on computer vision. They don’t offer much support for natural language processing (NLP) tasks, especially when it comes to languages other than English.

Regarding their products, CloudFactory acquired Hasty in 2022, a data-centric ML platform for computer vision applications. Hasty offers AI-powered image annotation, quality control, and a no-code model building solution for efficient application development. Additionally, CloudFactory provides “AI assistants” within Hasty to automate annotation tasks using data-specific training rather than generic pre-trained models.

Pricing Models

Dataset Types

iMerit Dataset Types

iMerit handles image and video datasets for computer vision tasks, including object detection, image segmentation, and image classification, which are essential for developing autonomous vehicles and facial recognition systems. iMerit also manages text and document datasets, providing services such as NLP, sentiment analysis, and entity recognition. Additionally, they work with audio datasets for speech recognition and transcription tasks.

CloudFactory Dataset Types

CloudFactory data labeling services are suitable for both computer vision data and NLP data. The data annotation formats they support include PNG Masks, JSON, and COCO for import, and COCO, Pascal VOC, JSON, and PNG Masks for export. The types of data the company works with are 2D image and video files, including PNG, JPG, WEBM, HEIC, BMP, tiff for images, and all video types supported by FFmpeg.

Data Annotation Tools

iMerit Annotation Tools

For the majority of annotations, iMerit experts use their own tool, Ango Hub, which is used for image, video, and text annotation. Even though they can use client’s individual tools, they prefer their own, and use it for all cases. iMerit technology is also available by subscription for individual annotation.

CloudFactory Annotation Tools

CloudFactory offers a platform alongside its data annotation services. This gives you the flexibility to use CloudFactory’s tools or integrate them seamlessly with their existing software. Their data analysts are tool-agnostic and can even learn custom tools to fit your ML project needs. So, CloudFactory can be a good fit for those owning their own tools but requiring assistance with scaling up labeling tasks.

We also discovered the vendor collaborates with other data labeling companies equipped with their own tool sets to offer a complementary workforce solution in addition to its own platform. Among their preferred tool partners are Dataloop, Datasaur.ai, and Labelbox.

For automation, CloudFactory offers a range of features, including label assistants, fully automated labeling, active learning, AI-consensus scoring, and additional automation features.

Integrations

iMerit Integrations

The company offers no-code integrations. The majority of deployment happens through APIs and plugins. They can integrate various applications and MLOps platforms. The data pipelines can be efficiently integrated even with custom requests.

CloudFactory Integrations

CloudFactory integrates with popular cloud storage platforms, such as AWS S3, Google Cloud Storage, and Azure Blob Storage for seamless data transfer. Additionally, they integrate with various ML frameworks (like TensorFlow and PyTorch) to streamline the model training workflow.

They offer a REST API for automating and outsourcing back-office data tasks, as well as programmatic management of labeling projects. This API integration allows developers to seamlessly connect data jobs to existing applications, streamlining workflow management.

Annotation Process

iMerit Annotation Process

Despite offering machine-assistant labeling for specific cases, the company believes human annotation is faster and more efficient. Here are their main steps for data annotation:

• Consultation with an expert

• Trial and annotators’ training

• Workflow customization

• Feedback cycle

• Evaluation

You may be involved in the annotation process at any stage you decide.

CloudFactory Annotation Process

Instead of having a traditional data labeling process, CloudFactory takes an integrated approach. Their annotation process incorporates a number of key steps.

1. Before you commit: Free analysis

2. Getting started with CloudFactory: Team onboarding

3. Data annotation process:

   • Data annotation

   • Quality assurance (QA)

   • Process iteration

   • Project management

In addition, the CloudFactory team provides a dedicated Client Success manager and Delivery Team Lead for each project, as well as a dedicated Channel Manager for ongoing support.

Quality Assurance

iMerit QA

During every step of data annotation, iMerit uses various reports, dashboards, and tracking systems to allow efficient project management, identify troubleshooting needs, and track KPI metrics. To ensure the quality of provided annotation, they use a combination of techniques:

• Setting a gold standard during mini-sets

• Using an annotator consensus

• Using scientific methods for label consistency

• Subsampling

The solution architect randomly chooses a sample of dataset, around 5-10% of the whole labeled data, and checks for possible errors. AI-based frameworks help annotators to review labeled datasets, which is done a couple of times before the project submission.

CloudFactory QA

Data analysts at CloudFactory use a combination of automated checks and human review to guarantee highly accurate annotations. Moreover, they claim a 100% QA guarantee.

Here’s what you need to know about data quality at CloudFactory:

• Built-in QA

• Model feedback

• Multi-layered quality control: CloudFactory uses a comprehensive approach to measure data labeling accuracy, including:

  • Gold Standard

  • Sample Review

  • Consensus

  • Intersection over Union (IoU)

Security and Data Compliance

TL;DR

In summary, both data annotation companies excel in data labeling but cater to different needs. iMerit is perfect for high-precision projects with its comprehensive services and stringent QA, albeit at a higher cost. CloudFactory offers flexible, budget-friendly solutions with fast turnaround, ideal for scalable tasks. Choose iMerit for detailed accuracy and CloudFactory for affordability and speed.

And if you’re looking for a vendor with these qualities:

• No commitment

• Flexible pricing

• Tool-agnostic

• Data-compliant

Then try running a free labeling pilot with us to quickly see if we’re a good fit.

FAQ

Written by

Yuliia Kniazieva Editor-at-Large

One of the technical writers at Label Your Data, Yuliia has been gradually delving into the intricate aspects of AI. With her strong passion for the written word and technical expertise, Yuliia has developed a keen interest in the evolving field of data annotation and the power of machine learning in today's tech-savvy world. Check out her articles to learn more about the complex world of technology and find the solutions that work best for your AI project!