Mike0307/ai-text-detector-zh-en

AI Text Detector (Chinese and English)

Detects whether a piece of Chinese or English text was written by a human or generated by AI, and estimates which model family it most resembles: ChatGPT, Claude, Gemini, or another AI. The model is xlm-roberta-base fine-tuned as a five-class classifier (human, openai, anthropic, google, other_ai). Its output is then calibrated into a single AI probability on a held-out validation set.

This repo contains ONNX files for in-browser inference. Text never has to leave the user's device.

Files

File	Purpose
onnx/model_fp16_ext.onnx + onnx/model_fp16_ext.onnx_data	fp16 model with weights in a separate file (about 390 MB). Used by the web app: with externalData, onnxruntime-web uploads weights straight to the GPU, which roughly halves peak memory during loading
onnx/model_fp16.onnx	The same fp16 model in a single file (about 390 MB)
onnx/model.onnx	fp32 weights for environments without fp16 support (about 780 MB)
onnx/model_int8_ext.onnx + onnx/model_int8_ext.onnx_data	Compact int8 model for phones and low-memory devices (about 200 MB), dynamically quantized per channel
fusion_int8.json	Calibration for the int8 model, refitted on the validation set
tokenizer.json	Original xlm-roberta tokenizer, unchanged
id_map.bin	Lookup table (int32, little-endian) that maps tokenizer ids to model ids; apply it before feeding the model
fusion.json	Calibration: features are [logit(1 - P(human)), is_chinese, log(1 + length)], standardized, then logistic regression
aidetect.json	Label order, max token length, segmentation and verdict parameters

To keep the download small, the embedding layer drops word pieces for scripts the model never sees in its task (Korean, Thai, Arabic, Cyrillic, Indic and others), going from 250,002 to 141,877 rows. The tokenizer itself is unchanged, so predictions on Chinese and English text match the original model. export_check.json records the comparison between the fp16 export and the original PyTorch model.

Usage

1. Tokenize the text with tokenizer.json (max length in aidetect.json).
2. Replace every token id with id_map[id].
3. Run the ONNX model on input_ids and attention_mask, then apply softmax to get the five class probabilities.
4. Compute the fusion features and apply fusion.json to get the calibrated AI probability.

Long documents are split into paragraphs and scored one segment at a time; aidetect.json holds the parameters.

Evaluation (v3)

Test set	AUROC	Accuracy	Human flagged as AI	AI detected
In-distribution test set (zh + en)	0.998	98.4%	2.6%	99.5%
English out-of-distribution (textsight 2026)	0.975	91.1%	9.3%	96.4%
Chinese out-of-distribution (DetectRL-X: GPT-4o, Gemini 2.5, DeepSeek, Qwen)	0.971	91.5%	10.0%	92.2%

The out-of-distribution sets come from sources outside the training data. The Chinese set uses generator models that were not part of training.

Compact model for phones

Mobile browsers cap the memory of a tab, and loading the fp16 model can take over 1.5 GB. The web app therefore uses the int8 model on phones, tablets and devices with 4 GB of memory or less. It is about half the size and needs roughly 1 GB to load. Its outputs drift slightly lower than fp16, so it ships with its own calibration fitted on the validation set (the test sets were not used).

Test set (fp16 / int8)	AUROC	Accuracy	Human flagged as AI	AI detected	Verdict differs from fp16
In-distribution test set	0.999 / 0.999	98.6% / 98.0%	2.2% / 3.2%	99.5% / 99.2%	1.4%
English out-of-distribution	0.977 / 0.966	91.4% / 90.2%	9.0% / 9.7%	96.4% / 89.3%	4.8%
Chinese out-of-distribution	0.971 / 0.970	91.6% / 91.1%	10.0% / 11.1%	92.4% / 92.2%	7.7%

Limitations

• Results are probability estimates. Do not use them as the sole basis for any disciplinary or legal decision.
• Short texts, formal and legal writing, classic literary prose, and English by non-native writers are more likely to be misclassified as AI.
• Source attribution is reliable only for model versions seen in training. It degrades on newer models and on AI text prompted to imitate a human style.
• Human training data only includes text that can be verified as written before 2022-11-30, the release date of ChatGPT.

License

MIT. The training data is not public.