CloudFactory vs. Kili Technology: In-Depth Review and Analysis

1. CloudFactory vs. Kili Technology: Company Profiles
   1. CloudFactory Company
   2. Kili Technology Company
2. Services and Products
   1. CloudFactory Services and Products
   2. Kili Technology Services & Products
3. Pricing Models
4. Dataset Types
   1. CloudFactory Dataset Types
   2. Kili Technology Dataset Types
5. Data Annotation Tools
   1. CloudFactory Annotation Tools
   2. Kili Technology Annotation Tools
6. Integrations
   1. CloudFactory Integrations
   2. Kili Technology Integrations
7. Annotation Process
   1. CloudFactory Annotation Process
   2. Kili Technology Annotation Process
8. Quality Assurance
   1. CloudFactory QA
   2. Kili Technology QA
9. Security and Data Compliance
10. TL;DR
11. FAQ

Struggling with inconsistent, low-quality labeled data that hinders your ML projects? Finding the right data labeling vendor is crucial yet challenging, as it directly impacts the accuracy and performance of your machine learning models.

This article compares two data labeling vendors, CloudFactory and Kili Technology, to help you navigate this critical decision.

CloudFactory vs. Kili Technology: Company Profiles

CloudFactory Company

CloudFactory offers human-in-the-loop (HITL) data labeling solutions, leveraging a global, on-demand workforce supported by AI technology. With a talent pool of over 7,000 data annotators, CloudFactory is trusted by more than 700 AI companies. Established in 2010 by Mark Sears, the company has a presence in the UK, US, Nepal, and Kenya. As of 2024, Kevin Johnston has taken on the role of CEO, with Sears transitioning to Executive Chairman.

Kili Technology Company

Kili Technology, founded in 2018 by Edouard d’Archimbaud and François-Xavier Leduc, is a data labeling platform designed for data scientists and engineers. Launched in 2020, it enables efficient labeling of large datasets for ML model training. Kili Technology supports various industries, including healthcare, finance, and retail. Major corporations like L'Oréal, Renault, and Airbus use the platform to improve AI applications such as facial recognition, autonomous driving, and predictive maintenance.

Services and Products

CloudFactory Services and Products

CloudFactory offers a wide range of human-led AI solutions, including data curation, annotation, quality assurance, and model optimization. While CloudFactory specializes mainly in computer vision data annotation, their support for natural language processing (NLP) is more limited, especially for languages other than English. Their key services consist of:

Data Labeling:

• Accelerated Annotation: AI-powered labeling for 2D images and videos, up to 30 times faster without sacrificing accuracy.

• Workforce Plus (workforce + tech): Comprehensive package for labeling video, LiDAR data, and more, with platform integration options.

• Vision AI Managed Workforce: Dedicated workforce trained for computer vision tasks.

• NLP: Text and audio data labeling using their platform or yours.

• Data Processing: Workforce support for business process optimization and back-office tasks.

Human-in-the-Loop Automation:

• Managed Workforce: Skilled workforce complementing AI automation, mainly for computer vision tasks in industries like aerial and geospatial, autonomous vehicles, finance, healthcare, insurance, and retail.

Products:

• Hasty: A data-centric ML platform for computer vision applications acquired in 2022. It offers AI-powered image annotation, quality control, and no-code model building.

Kili Technology Services & Products

Kili Technology provides comprehensive data labeling solutions to help businesses and data scientists build high-quality machine learning datasets. Their core offerings include a data labeling platform, professional services with a global annotator workforce, and expert guidance from Machine Learning Engineers (MLEs).

Kili Data Labeling Platform:

• Supports labeling for text, documents, images, and videos

• Offers AI-assisted tools to enhance manual labeling

• Key features: labeling tools, quality management, integration capabilities, LLM fine-tuning, evaluation, and testing

Professional Services:

• Managed Expert Labeling Service: Expert-led data labeling for high-quality datasets

• Kili Simple: Global annotators for large-scale tasks across various formats

• ML Expert Guidance: Access to API, quality control, customizable annotation, and real-time project oversight

• Professional Services: Consulting by MLEs for project assessment and implementation

Products:

• davinci: A generative AI tool for patent drafting and office action response, distinct from similarly named products despite trademark disputes.

Pricing Models

Dataset Types

CloudFactory Dataset Types

CloudFactory services are suitable for both computer vision data and NLP data. The data annotation formats they support include PNG Masks, JSON, and COCO for import, and COCO, Pascal VOC, JSON, and PNG Masks for export.

The types of data the company works with are 2D image and video files, including PNG, JPG, WEBM, HEIC, BMP, tiff for images, and all video types supported by FFmpeg.

Kili Technology Dataset Types

Kili Technology's platform simplifies the annotation of various unstructured data formats, including images, videos, text documents, PDFs, satellite imagery, and conversational data. It offers specialized interfaces and features for efficient and accurate labeling, supporting LLM evaluation, supervised fine-tuning, and RLHF workflows.

Data Transfer and Formats:

• Import Formats: CSV, JSON, image files

• Export Formats: CSV, JSON, TensorFlow Record

Platform Limitations:

• Video Annotation: Supports only bounding boxes

• OCR Integration: Requires users to upload their own OCR data for text annotation in images

• Supported Formats: Common image formats are accepted, but Excel and Word documents need transformation before import.

Data Annotation Tools

CloudFactory Annotation Tools

CloudFactory provides a versatile platform for data annotation, allowing clients to either use their tools or integrate with existing software. Their data analysts are adaptable, capable of learning custom tools to meet specific ML project requirements, making CloudFactory ideal for scaling up labeling tasks.

Key Features:

• Flexibility: Use CloudFactory’s tools or integrate with your own

• Tool-Agnostic Analysts: Capable of learning custom tools

• Partnerships: Collaborates with data labeling companies like Dataloop, Datasaur.ai, and Labelbox for complementary workforce solutions

Automation Features:

• Label assistants

• Fully automated labeling

• Active learning

• AI-consensus scoring

• Additional automation features

Kili Technology Annotation Tools

Kili Technology’s platform allows you to interact with pre-labeled data, making necessary adjustments to ensure high-quality labels, even with automation. You can use custom models for pre-labeling, and the platform employs built-in AI models like ChatGPT and SAM to automatically pre-label raw data, saving time for large datasets.

If you prefer the team to use your specific tool, this option is available for an additional fee. With their extensive experience, the specialists can also provide feedback on your tool and suggest any necessary improvements.

Annotation Tools:

• Text Annotation Tool

• Image Annotation Tool

• Video Annotation Tool

• OCR Annotation Tool

• Geospatial Annotation Tool

Key Features:

• Interaction with pre-labeled data for quality adjustments

• Custom model integration for pre-labeling

• Built-in AI models for automatic pre-labeling

Drawbacks:

• Automation features are still under development, requiring human annotators for complex tasks

• Focus on proprietary tools may lead to integration challenges with existing labeling tools, potentially hindering a seamless workflow.

Integrations

CloudFactory Integrations

CloudFactory integrates with major cloud storage platforms, including AWS S3, Google Cloud Storage, and Azure Blob Storage, ensuring seamless data transfer. They also support various machine learning frameworks like TensorFlow and PyTorch to streamline model training workflows.

Key Integration Features:

• Cloud Storage: AWS S3, Google Cloud Storage, Azure Blob Storage

• Machine Learning Frameworks: TensorFlow, PyTorch

• REST API: Automates and manages labeling projects and back-office data tasks programmatically

• Data Security: Prioritizes secure cloud environments

• Data Ownership: Users retain full and exclusive ownership of all uploaded data

These integrations enable developers to connect data jobs to existing applications, enhancing workflow management and efficiency.

Kili Technology Integrations

Kili Technology offers a robust suite of tools to manage data quality, enhance collaboration, and integrate seamlessly into existing machine learning workflows.

Integration Tools:

• API and Python SDK: Enable programmatic access to core functionalities, allowing users to manage data quality and integrate Kili into ML pipelines.

• Data Storage and Cloud Platforms: Supports Amazon S3, Google Cloud Storage, and Microsoft Azure Blob Storage, simplifying data import and export. This allows for direct initiation of labeling tasks without manual data transfer.

• Version Control: Facilitates tracking of data changes and export versions in preferred model formats (e.g., YOLO, PASCAL VOC).

Simplified Labeling Operations Management:

• Integration with existing ML stacks, datasets, and large language models (LLMs)

• Easy import/export of data

• Management of labeling projects

• Oversight of the entire training data lifecycle within Kili

Key Considerations:

• Full functionality requires the use of webhooks, API access, and the Python SDK.

• The API might have a steeper learning curve for users without technical expertise.

Annotation Process

CloudFactory Annotation Process

CloudFactory integrates human expertise, established workflows, and advanced technology for data labeling. Here’s an overview:

Pre-Commitment:

• Free analysis: Cloudfactory reviews your instructions, tests tasks, and offers feedback. This 10-hour mini pilot labels a sample set and provides a detailed report with recommendations.

Getting Started:

• Team Onboarding: A dedicated team is onboarded over two weeks, determining expert needs and recruiting additional workers if necessary. Their global network handles large projects efficiently.

Data Annotation Process:

• Data Annotation: The team labels data per your specifications.

• Quality Assurance: Rigorous quality checks at every step.

• Process Iteration: Continuous monitoring and adjustments to refine workflows and QA procedures. The team size can be scaled as needed.

• Project Management: Comprehensive project planning, implementation, and measurement ensure desired outcomes.

Support and Management:

• Dedicated Support: Each project has a Client Success Manager, Delivery Team Lead, and Channel Manager for ongoing support.

Kili Technology Annotation Process

Kili Technology offers versatile data labeling services essential for any ML project, with comprehensive tools tailored to streamline the annotation process based on your specific needs.

Platform-Based Annotation:

• Quick Start: Begin annotating data quickly with an intuitive platform.

• Customization: Configurable interfaces and pre-annotation using your models expedite labeling.

• Quality Assurance: Integrated quality metrics identify and address errors, ensuring high-quality datasets.

• Labeling Job Types: Supports classification, object/entity detection, object/entity relation, and transcription.

• Process:

  • Design project-specific interfaces.

  • Assign data to annotators, set validation rules, and use segmentation/tracking tools.

  • Pre-label data with model predictions.

  • Analyze productivity and review critical data points with advanced metrics.

• Integration: Seamlessly integrates with Amazon, Google, and Microsoft cloud storage.

• Export: Export data in your model’s format and manage access rights with predefined roles.

• Automation: API and Python SDK integration for a smooth ML stack connection, with webhooks and plugins for MLOps automation.

Kili Simple Annotation Process:

• Data Submission

• Project Initialization

• Real-Time Monitoring

• Delivery

ML Expert Guidance:

• Requirement Communication

• Feedback Loop

• Progress Monitoring

• Continuous Improvement

Quality Assurance

CloudFactory QA

CloudFactory ensures highly accurate annotations through a combination of automated checks and human review, boasting a 100% QA guarantee. Here's an overview of their quality assurance process:

1. Built-in QA: Quality checks are integrated throughout the workflow, as specified by your SLA.

2. Model Feedback: For computer vision tasks, CloudFactory provides feedback to improve your model, not just the data labeling.

3. Multi-layered Quality Control:

   • Gold Standard

   • Sample Review

   • Consensus

   • Intersection over Union (IoU)

Kili Technology QA

Ensuring high-quality annotations is essential for successful data annotation projects. Kili Technologies offers a range of tools and processes to enhance quality management.

QA Tools and Processes:

• Annotation Consistency Checks: Automated checks review consistency across multiple annotators, identifying and resolving discrepancies.

• Review and Feedback: Annotations can be reviewed by team members, and feedback is provided in an iterative loop for continuous improvement.

• Quality Control Metrics: Metrics such as inter-annotator agreement and annotation speed are used to assess annotator performance and identify areas for improvement.

Kili Technologies guarantees 95% accuracy based on specific project requirements.

Approaches to Data Labeling Quality:

• Low-Error Datasets

• Data Cleaning

• Programmatic QA

Security and Data Compliance

TL;DR

CloudFactory and Kili Technology both excel in providing high-quality data labeling services but cater to different needs. CloudFactory offers scalable HITL solutions and emphasizes a robust global workforce, making it ideal for large-scale computer vision projects. In contrast, Kili Technology provides a versatile platform with strong AI-assisted tools and seamless integration capabilities, making it suitable for diverse data types and detailed quality management.

While CloudFactory’s flexible pricing and comprehensive QA processes ensure accuracy and scalability, Kili’s tiered pricing and advanced annotation tools offer tailored solutions for data scientists seeking precision and efficiency.

But if you need a data labeling vendor that offers:

• No commitment

• Flexible pricing

• Tool-agnostic

• Data-compliant

Run a free pilot to put our labeling expertise to the test.

FAQ

by Yuliia Kniazieva
on July 9, 2024. Share on LinkedIn Post on Twitter