IT SOLUTION

IT Services

We select the most suitable personnel from a pool of over 8,000 testers
based on their individual interests and areas of expertise.
In addition, we conduct diagnostics
of target training data and annotation QA.

Recommended for the Following Customers

  • Generated information contains numerous errors

  • Answers reflect a specific viewpoint or bias in opinion

  • Testing on devices covering over X% of the domestic smartphone market share

  • Expressions and ideas are limited and lack creativity

Training Data Assessment IT SOLUTION

By examining only a portion (%) of the total training data,
the confidence score of the entire dataset can be estimated.

Flowchart: Flow of training data diagnostics. Step 1: Select the target sample size → Step 2: Perform data diagnosis (diagnosis of annotated data) → Step 3: If the error rate in the diagnosis is below 5%, the results are considered to have a high confidence score and suitable for drawing conclusions and use in continued research. If the error rate is 5% or higher, the results are considered to have a low confidence score. → Depending on the result, annotation QA is recommended. For reference, increasing training data barely changes the sample size required for a 95% confidence score (10,000 → 370, 100,000 → 383, 1,000,000 → 384, 10,000,000 → 385).
Reference

The sample size required for 95% confidence hardly fluctuates due to the amount of training data

Number of training data Sample size
10,000 370
100,000 383
1,000,000 384
10,000,000 385

Improvement Process

  1. 1

    Analyze NGs by cause type

    The analysis results are shared with the annotation company through the client to help with subsequent improvements

    Item Number of NGs Occurrence rate
    Property error 360 6.01%
    Annotation omission 2 0.04%
    Class error 18 0.34%
  2. 2

    Conduct analysis continuously

    Based on the analysis results, repeat step ① regularly until improvements are made, and observe the trend of the occurrence rate

    Item Number of NGs Occurrence rate
    Property error 360 6.01%
    Annotation omission 2 0.04%
    Class error 18 0.34%
    Occurrence rate
    0.99%
    0.04%
    0%
  3. 3

    Set key points

    Once the pre-set target values are achieved, move on to the brush-up phase. Set key points according to the AI's intended use and focus on reducing the occurrence rate of the relevant items

    Item Number of NGs Occurrence rate
    Property error 10 1.72%
  4. 4

    Achievement of target values for key points

    Regularly observe the trend of the occurrence rate. Annotation QA is complete once the target values are achieved

    Item Number of NGs Occurrence rate
    Property error 1 0.17%
Consult about data diagnostics

Other Services SERVICE

  • Software Testing

    • Functional testing for various types of software, primarily web services
    View Details

  • Automotive System Verification

    • Data verification for autonomous driving
    • Various driving tests (motion sickness data acquisition driving, emotion data acquisition driving, etc.)
    • Car navigation test drive
    • Tests for car accessories (digital mirrors, etc.)

    (Excluding embedded systems)

    View Details
Back to Business Software Top