A Comprehensive CloudFactory Review

1. How to Choose a Dataset Labeling Vendor?
2. CloudFactory Review
3. CloudFactory Services & Products
4. CloudFactory Dataset Types
5. CloudFactory Data Annotation Tools
6. CloudFactory Integrations
7. CloudFactory Annotation Process
8. CloudFactory Quality Assurance
9. CloudFactory Pricing
10. CloudFactory Security and Data Compliance
11. Top CloudFactory Alternatives
    1. Label Your Data
    2. iMerit
    3. Humans in the Loop
12. FAQ

As we’re getting more dependent on AI solutions, data labeling becomes key to building powerful machine learning models. Yet, AI engineers struggle to balance rapid model deployment with the need to label data manually.

This blog dives deep into CloudFactory company review, a prominent player in the data labeling space, to help you decide if it’s the perfect match for your ML project.

How to Choose a Dataset Labeling Vendor?

Machine learning projects require massive amounts of labeled data for training and testing the models. There are two main approaches to get the training data: tackling it internally or outsourcing the task.

Finding the right data labeling vendor for text, audio, or image annotation tasks can feel like searching for a needle in a haystack. If you choose to outsource and consider CloudFactory as your potential vendor, you’re likely interested in learning more about them before committing. To save you valuable time, potentially hours of research, we’ve analyzed the vendor and made this CloudFactory review for you.

Here are some key factors to consider when choosing a data labeling vendor:

• Service and products

• Dataset types

• Data annotation tools

• Integrations

• Annotation process

• Quality assurance

• Pricing models

• Security and data compliance

Now, we’ll look at each factor in more detail.

CloudFactory Review

CloudFactory provides human-in-the-loop (HITL) data labeling solutions for ML projects. The company leverages a global, on-demand workforce alongside AI-assisted technology to deliver high-quality datasets. Over 700 successful AI companies trust CloudFactory’s services, powered by a large talent pool of more than 7,000 highly trained data analysts.

Founded in 2010 by Mark Sears in Kathmandu, Nepal, CloudFactory offers meaningful work opportunities in developing countries, while simultaneously helping businesses automate tasks. With headquarters in Kowloon, China, and offices spanning four continents (UK, US, Nepal, and Kenya), CloudFactory serves major players across various sectors like healthcare, retail, finance, and geospatial.

*As of January 2024, CloudFactory underwent a leadership change, with Kevin Johnston assuming the CEO role and Sears transitioning to Executive Chairman.

CloudFactory Services & Products

By bringing together people and technology, CloudFactory delivers human-in-the-loop AI solutions, from data curation and annotation, to QA and model optimization. As part of our CloudFactory company review, we’ve analyzed their core services, including:

Data Labeling

• Accelerated Annotation: CloudFactory’s “flagship data labeling product,” in which the company offers AI-powered labeling for 2D images and videos, achieving up to 30 times faster labeling speeds without sacrificing accuracy.

• Workforce Plus (workforce + tech): A complete package for labeling video, LiDAR data, and more, along with the tools you need. You can use their platform or even integrate it with your own.

• Vision AI Managed Workforce: CloudFactory offers a dedicated workforce trained specifically for computer vision tasks (which they call Vision AI).

• NLP: Need help with text or audio data labeling? CloudFactory’s Workforce can handle that too, using their platform or yours.

• Data Processing: CloudFactory’s Workforce goes beyond just data labeling. They can also help you optimize your business processes with data processing and other back-office tasks.

Human-in-the-Loop Automation

• Managed Workforce: CloudFactory offers a skilled workforce that complements AI automation.

While CloudFactory can identify areas for improvement in training data, their services are mostly focused on labeling data for computer vision. They serve industries, such as aerial and geospatial, autonomous vehicles, finance, healthcare, insurance, and retail. However, the CloudFactory team doesn’t offer much support for natural language processing (NLP) tasks, especially when it comes to languages other than English.

Don’t limit your NLP project — get multilingual data labeling in 55 languages. Get in touch with us!

Regarding their products, CloudFactory acquired Hasty in 2022. It’s a data-centric ML platform specifically designed for creating and deploying computer vision applications. This machine-learning software offers features like AI-powered image annotation for creating ground-truth datasets, quality control, and no-code model building solution to ensure your applications are built efficiently and accurately.

The computer vision tasks the platform can handle:

• Classification

• Tagging

• Object Detection

• Instance Segmentation

• Semantic Segmentation

• Panoptic Segmentation

• Attribute Prediction

Besides, CloudFactory offers AI tools called “AI assistants” within Hasty to automate some or even all of your annotation tasks. They are trained specifically on the data you provide, instead of relying on generic pre-trained models. Our CloudFactory overview provides a good starting point, but for a deeper dive into the platform, explore their Computer Vision Wiki.

CloudFactory Dataset Types

CloudFactory services are suitable for both computer vision data and NLP data. The data annotation formats they support include PNG Masks, JSON, and COCO for import, and COCO, Pascal VOC, JSON, and PNG Masks for export.

The types of data the company works with are 2D image and video files, including PNG, JPG, WEBM, HEIC, BMP, tiff for images, and all video types supported by FFmpeg.

For Accelerated Annotation, the company offers the following dataset labeling types:

• Image Classification

• Object Detection

• Instance Segmentation (Polygons + Pixel Masks)

• Semantic Segmentation (Polygons + Pixel Masks)

• Panoptic Segmentation (Polygons + Pixel Masks)

• Keypoint Annotation

• Video Understanding

For Managed Workforce, you get to choose from these Computer Vision labeling types:

• Bounding Boxes

• Landmarking

• Wireframe

• Masking

• 3D Cuboid

• Polygon

• Polyline

• Object Tracking

• Transcription

Human-in-the-Loop Automation data labeling types supported:

• Exceptions, Errors, and Edge Cases

• Manual Workflows

• Transcription

CloudFactory Data Annotation Tools

CloudFactory offers a platform alongside its labeling services. This gives you the flexibility to use CloudFactory’s tools or integrate them seamlessly with their existing software. Their data analysts are tool-agnostic and can even learn custom tools to fit your ML project needs. So, CloudFactory can be a good fit for those owning their own tools but requiring assistance with scaling up labeling tasks.

Digging deeper into the CloudFactory company overview, we discovered the vendor collaborates with other data labeling companies equipped with their own tool sets to offer a complementary workforce solution in addition to its own platform. Among their preferred tool partners are Dataloop, Datasaur.ai, and Labelbox.

For automation, CloudFactory offers a range of features, including label assistants, fully automated labeling, active learning, AI-consensus scoring, and additional automation features.

CloudFactory Integrations

CloudFactory integrates with popular cloud storage platforms, such as AWS S3, Google Cloud Storage, and Azure Blob Storage for seamless data transfer. Additionally, they integrate with various machine learning frameworks (like TensorFlow and PyTorch) to streamline the model training workflow.

They offer a REST API for automating and outsourcing back-office data tasks, as well as programmatic management of labeling projects. For user assurance, the company prioritizes data security within its secure cloud environment. Additionally, users retain full and exclusive ownership of all uploaded data. This API integration allows developers to seamlessly connect data jobs to existing applications, streamlining workflow management.

CloudFactory Annotation Process

Now to the main part of our CloudFactory overview. Instead of having a traditional data labeling process, CloudFactory takes an integrated approach. This means they view labeling as a combination of human expertise, established workflows, and powerful technology. To achieve this, their annotation process incorporates a number of key steps.

Before you commit:

• Free analysis: They don’t offer a traditional free trial, but they do provide a complimentary “analysis” of your project, wherein CloudFactory reviews your instructions, tests some tasks with their team, and offers feedback on how to improve your labeling approach. It essentially functions as a mini pilot project, taking around 10 hours to complete. This analysis helps them assess your data, label a sample set, and identify the best quality assurance procedures for your project. They also ensure your instructions are clear and provide a detailed report with recommendations.

Getting started with CloudFactory:

• Team onboarding: They take two weeks to onboard a dedicated team for your project. This onboarding period helps them determine the number of experts required to meet your service level agreement (SLA). If necessary, they’ll also recruit additional workers during this time. CloudFactory leverages a network of distributed annotators worldwide, allowing them to handle large projects efficiently. Their cloud-based platform connects businesses with these skilled workers.

Data annotation process:

• Data annotation: Once the team is ready, they’ll begin labeling your data according to your specifications.

• Quality assurance (QA): Every step involves rigorous quality checks to ensure accuracy.

• Process iteration: They monitor the process and make adjustments as needed. This could involve refining data features, adapting task workflows, or enhancing QA procedures. Also, CloudFactory can easily scale the team up or down by hiring or reassigning annotators as needed.

• Project management: They handle project planning, process implementation, and ongoing measurement to ensure your project meets the desired outcomes.

In addition, the CloudFactory team provides a dedicated Client Success manager and Delivery Team Lead for each project, as well as a dedicated Channel Manager for ongoing support.

Labeled datasets aren't accurate enough? Contact our expert team!

CloudFactory Quality Assurance

Data analysts at CloudFactory use a combination of automated checks and human review to guarantee highly accurate annotations. Moreover, they claim a 100% QA guarantee.

Here’s what you need to know about data quality at CloudFactory:

• Built-in QA: They integrate quality checks throughout the workflow as specified by your SLA.

• Model feedback: They provide feedback on how to improve your model, not just the data labeling itself. Yet, this applies only to computer vision tasks.

• Multi-layered quality control: CloudFactory uses a comprehensive approach to measure data labeling accuracy, including:

  • Gold Standard: When there’s a single correct answer, they measure quality based on the number of correctly and incorrectly labeled tasks.

  • Sample Review: A more experienced worker (team lead or project manager) from CloudFactory reviews a randomly selected sample of completed tasks for accuracy.

  • Consensus: Multiple workers are assigned the same task, and the answer chosen by the majority is considered correct.

  • Intersection over Union (IoU): This method combines human input with automation to compare the bounding boxes in your ground truth images with the predicted bounding boxes generated by your model.

CloudFactory Pricing

Delving further into CloudFactory review, the company has flexible pricing plans that can adjust to your ML project needs. Here’s a short breakdown:

• Computer vision tasks: You pay per object (e.g., bounding box) you need labeled.

• NLP tasks: You are charged by the hour.

They also have a yearly agreement option with a fixed cost, but you’ll be billed monthly.

When using their Accelerated Annotation, you’ll get:

• A team and tools all set up for you

• Top-notch AI assistance

• Feedback on how well your model is doing

• Double-checking everything with AI and human experts

• Pay only for what you use

This option supports only Computer Vision tasks.

If you choose their Workforce Service, you’ll pay for:

• Skilled people to analyze your data

• Training focused on your specific needs

• The Client Success support

• Prices that can change based on your needs

• Discounts for high-volume projects

• The option to use your own tools or theirs

This pricing option covers Computer Vision, NLP, and Data Processing tasks.

CloudFactory Security and Data Compliance

Last but not least, we wanted to take a deeper focus on data compliance as part of our CloudFactory company overview. The company follows the highest industry standards and has certifications to prove it, including ISO 9001:2015, ISO 27001:2013, SOC 2, HIPAA, and GDPR. They also use a platform called OneTrust to track the services and vendors they use and how they process and store data throughout the company.

To keep your information safe, CloudFactory uses a secure browser to access work and makes sure all worker computers have antivirus software. All people involved in your project are required to sign a security agreement and a non-disclosure agreement (NDA).

Furthermore, CloudFactory uses a secure network environment (SNE) with advanced threat protection (ATP) to keep your data safe. This includes anti-malware, intrusion prevention, and IP whitelisting and web filtering.

Top CloudFactory Alternatives

After completing this CloudFactory company review, you may want to explore alternative data labeling companies on the market. There are 3 alternatives for you to consider:

Label Your Data

At Label Your Data, we help data scientists and operation managers streamline their dataset labeling flow. Our global workforce delivers high-quality labeled data for computer vision and NLP projects. Having completed over 100 projects, we offer agile solutions to streamline your model training, including a free pilot, adaptable pricing models, and tool-agnostic teams. Most importantly, our team adheres to industry-standard certifications like PCI DSS Level 1, ISO:2700, GDPR, CCPA.

Run free pilot!

iMerit

iMerit offers comprehensive solutions for computer vision and NLP tasks, as well as generative AI training data solutions and content services. The company served a wide range of clients across diverse fields like self-driving cars, medical diagnosis, and smart agriculture. They have a global presence with headquarters in the US and specialist teams located in India, US, Bhutan, and Europe. Yet, the key factor to consider is that iMerit offers a variety of end-to-end labeling solutions, all while adhering to strict data security regulations. You can discover more about the vendor in our iMerit review.

Humans in the Loop

Humans in the Loop offers dataset collection and annotation services, HITL for active learning, real-time edge case handling, and reinforcement learning with human feedback. However, they specialize only in computer vision annotation tasks for various industries, from self-driving cars to medical imaging. In addition, the company provides free ML datasets. And to ensure a smooth workflow, they partner with leading data annotation platforms (including Hasty).

FAQ

by Yuliia Kniazieva
on March 28, 2024. Share on LinkedIn Post on Twitter