Start Free Pilot

fill up this form to send your pilot request

Email is not valid.

Email is not valid

Phone is not valid

Some error text

Referrer domain is wrong

Thank you for contacting us!

Thank you for contacting us!

We'll get back to you shortly

TU Dublin Quotes

Label Your Data were genuinely interested in the success of my project, asked good questions, and were flexible in working in my proprietary software environment.

Quotes
TU Dublin
Kyle Hamilton

Kyle Hamilton

PhD Researcher at TU Dublin

Trusted by ML Professionals

Trusted by ML Professionals
Back to blog Back to blog
Published June 6, 2024

iMerit vs. Humans in the Loop: Who Is Your Next Choice?

iMerit vs. Humans in the Loop: Comparing Data Labeling Providers

As new machine learning projects continue to emerge, the importance of preparing accurate data for them is higher than ever before. Data annotation services deal with this task, making a critical contribution to AI development.

That's why we have prepared for you a comparative analysis of iMerit and Humans in the Loop, two leading data labeling companies in the market. We've analyzed their main data annotation services, pricing, and more to save you time on research. Check if they have what you're looking for.

iMerit vs. Humans in the Loop: Company Profiles

Feature

iMerit

Humans in the Loop

Founded

2012

2017

Headquarters

San Jose, California

Sofia, Bulgaria

Market Focus

  • Autonomous vehicles

  • Medical AI

  • Geospatial technology

  • Financial services

  • Commerce

  • Government

  • Agricultural AI

  • Geospatial technology

  • Automotive

  • Medical

  • Industrial

  • Agricultural

  • Retail

iMerit Company

iMerit was founded in 2012 by Dipak Basu. Today with over 5000 employees, the company offers full end-to-end services for data annotation in a wide array of industries. The company is headquartered in San Jose, California, but also has offices in New Orleans, LA, Kolkata, and Bengaluru, India.

Humans in the Loop Company

Humans in the Loop is a social enterprise which provides ethical dataset annotation and AI model validation services. Its data labeling services are provided by over 250 specialists located in Sofia, Bulgaria, their headquarters, but also Malaysia and Ukraine, Turkey, and the Middle East. One of its missions is to help people from countries that experience armed conflicts and those forced to displace.

Services and Products

iMerit Services and Products

iMerit's core services

In comparison to other data labeling companies that offer a limited number of services, the iMerit company works with both texts and images. It provides data labeling for machine learning, computer vision, natural language processing (NLP), as well as content services.

Their main services include:

  • Reinforcement learning from human feedback (RLHF) for LLMS and LVMS

  • Image annotation

  • Video annotation

  • Text annotation

  • Audio transcription

  • LiDAR annotation

  • Sentiment analysis

  • Content moderation

  • Product categorization

  • Image segmentation

Besides, you can receive data labeling and annotation services in various languages. In addition to English, their annotators based in Latin America, Germany, US, and Bhutan, can support other languages on request.

Humans in the Loop Services and Products

Humans in the Loop's core services

Humans in the Loop is a data labeling company that mostly specializes in computer vision. Their main services include:

  • Dataset annotation

  • Dataset collection

  • Human-in-the-loop for active learning

  • Real-time edge case handling

  • Reinforcement learning with human feedback

If your project is based on computer vision, Humans in the Loop can not only annotate the data for you. They can also collect it, review your final models, and assist with their testing before final deployment.

Pricing Models

iMerit's core pricing components

Feature

iMerit

Humans in the Loop

Pricing Structure

Subscription-based, per task, discounts for high volume

Depending on the task, per task or per hour

Pricing Tiers

  • Monthly subscription for Ango Hub platform

  • Universal annotation task pricing varies by dataset volume

  • Discounts for more than 100k objects per month

  • One-time charge for custom output export

  • Individual calculation of price

  • Dependent on volume, timelines, and difficulty of the project

Free pilot

Yes

Yes

Additional Notes

  • Discounts based on annotator's language and location

  • Pricing flexibility depending on the scope of annotation

  • Available for express/urgent annotation

Among many data annotation companies in the market, Humans in the Loop's free trial is counted per hour and not volume. With 2 hours of free annotation, both you and the company will be ready to estimate further timelines and cost of the annotation accomplishment.

Dataset Types

iMerit Dataset Types

iMerit technology services include various dataset types. They range from datasets for text annotation to video and image annotation. Besides, they transform audio data into text formats that can be further processed by computers. The supported formats are then used for named entity recognition and conversation categorization tasks.

Humans in the Loop Dataset Types

While labeling data for machine learning projects, Humans in the Loop work with the majority of acceptable dataset types. Among the most popular ones they mention JSON, Yolo, COCO, and Pascal Voc XML. However, they can also consider and find solutions for your specific format, both for importing and exporting data.

Data Annotation Tools

iMerit

Humans in the Loop

Tool (ownership)

Proprietary tool

Open source, third-party tools

Name

Ango Hub

Alegion, Diffgram, Human Lambdas, Hasty, Kili Technology, Lightly, V7, Manthano, Superannotate, Segments.ai

Other notes

Used for image, video, and text annotation

Can also work on your own tool

It's important to note that both companies adhere to a humans-in-the-loop approach, which means that behind every annotation there is a human. Even though both use various tools for annotation, it's a human annotator who processes data, annotates, and validates it.

Integrations

Both iMerit and Humans in the Loop offer no-code integrations. The final annotated data can be deployed to you through APIs. While iMerit technology also considers working with plugins and proceeds with custom requests, Humans in the Loop offers you the possibility to indicate the URLs where you want to receive final annotated datasets.

Annotation Process

iMerit Annotation Process

iMerit's annotation process is comprehensive and consistent, where you can participate at any stage. The main annotation steps include:

  • Consultation with an expert and registration on the platform

  • Trial and annotators’ training

  • Workflow customization

  • Feedback cycle

  • Evaluation

iMerit is one of a few AI data labeling companies that offers a trial with their Ango Hub platform.

Humans in the Loop Annotation Process

Humans in the Loop's perks

If you choose Humans in the Loop for your ML journey, the annotation with them will go through the following steps:

  • Introduction call

  • Free trial

  • Fine-tuning

  • Contract conclusion

  • Project delivery

Quality Assurance

iMerit QA

iMerit uses multiple layers of quality assurance along the whole data annotation process. To better notice the gaps, identify the efficiency of the accomplishment, and track KPI metrics, they use various dashboards, reports, and tracking systems. Their techniques for ensuring data annotation quality include:

  • Setting a gold standard during mini-sets

  • Using an annotator consensus

  • Using scientific methods for label consistency

  • Subsampling

Around 5-10% of random annotated data is additionally checked by a solution architect. Besides, data annotators review the labeled datasets multiple times before the final submission of the project.

Humans in the Loop QA

As the majority of data annotation companies, Humans in the Loop checks the annotated datasets multiple times. Their two levels of QA consist of:

  • Supervisor dedicated to every project

  • QA team

While the supervisor checks every piece of the annotated data and checks it against previously defined acceptance rates, the QA team checks the whole project at the end before sending it to the client.

Security and Data Compliance

Feature

iMerit

Humans in the Loop

Data Encryption

-

  • All datasets are encrypted at rest.

  • All applications encrypt in transit with TLS/SSL only.

Access Controls

  • Security Manager sets and monitors security measures

  • Role-based access

  • Role-based access

  • Quarterly access reviews

  • SSO and 2FA access to cloud services

Worker Screening

Security Manager trains employees

  • Background checks

  • All workers sign security and NDA agreements

Compliance

  • SOC 2

  • ISO 27001

  • ISO 9001:2015

  • GDPR

  • TISAX

  • GDPR

  • Data Protection Policy, in accordance with the Regulation (EU) 2016/679

  • SOC 2

iMerit vs. Humans in the Loop: Pros and Cons

To let you make your own decision whether the two data annotation providers fall under your top data labeling companies, here is the main summary:

Depending on the nature and size of your project, both iMerits' and Humans in the Loop AI solutions may be helpful:

Aspect

iMerit Pros

iMerit Cons

Humans in the Loop Pros

Humans in the Loop Cons

Services

Fuller range of services for images, videos, audio, and texts

The majority of services are provided in English. Other languages are possible on demand only, and not all

In addition to data annotation, there are also data collection, testing, and training services for ML models

Focus on computer vision

Tools

Own tool, which influences also security

Subscription-based tool

Possibility to manage multiple formats and datasets

Annotating with third-party and open-source tools

QA

More techniques for QA

Time-consuming QA processes

Quick turnaround of project tasks

Less stringent QA and potential errors

TL;DR

To summarize, both iMerit and Humans in the Loop offer competitive solutions. If the first vendor may be interesting to those who consider comprehensive annotation, the second one may suffice to those focusing on computer vision projects.

iMerit may be suitable to those who want to have extensive cooperation with regular subscription to their platform. Humans in the Loop is a good option for smaller projects that consider annotation, but also testing and training.

And if you're looking for someone who has experience with other industries (e.g. insurance, academia) or provides annotation in multiple languages across the globe, try a free pilot with us.

FAQ

For data annotation, which company offers a better combination of experience and cost: iMerit or Humans in the Loop?

Both iMerit and Humans in the Loop have extensive experience across various industries. They work with different dataset types and have a tailored approach to every client. While Humans in the Loop have had experience with geospatial, automotive, medical, retail industries, to name just a few, iMerit additionally covers commerce and finances. Both have individual pricing per project, but iMerit also considers subscriptions for their Ango Hub platform.

How do iMerit and Humans in the Loop compare in terms of data labeling accuracy and turnaround times?

iMerit has an elaborated QA that consists of multiple quality check techniques. This ensures the final annotated dataset is checked several times and has a high level of accuracy. Humans in the Loop, in their turn, have dedicated specialists and teams that check the annotated data after every task and at the end of the project.

If my project requires a significant increase in labeling volume, which company, iMerit or Humans in the Loop, can better scale their workforce to meet my needs?

iMerit counts over 5000 specialists, while Humans in the Loop have only around 250. This means that iMerit has more capacity for scaling and considering high-volume projects. They also offer discounts, creating attractive conditions. At the same time, Humans in the Loop are also ready to undertake express requests, although their price may increase.

Which company offers more responsive customer support?

Humans in the Loop assign you a project coordinator who serves as an intermediary between you and the annotators. Not only does he take your requests, but also ensures the quality of annotation. Besides, you can intervene at any stage and ask for additional reports from the team. iMerit also has a responsive team. However, although you can also comment and respond on various annotation stages, you communicate through the platform, which may delay the communication time.

Written by

Maria Shepeta
Maria Shepeta Editor-at-Large

Maria is one of the passionate tech writers on Label Your Data's content team. She enjoys combining the art of words with technological innovations. Interested in hot topics such as machine learning, artificial intelligence, and data annotation, she contributes to our knowledge base. Read some of her articles to discover evolving tech topics!