Leading autonomous AI scientist systems, evaluated head-to-head on the AI-READI multimodal diabetes dataset.
End-to-end research pipeline — hypothesis, experiments, and manuscript with no human steps.
Specialist agents coordinated across the full research lifecycle on complex biomedical data.
Open-source autonomous scientist covering all stages from hypothesis through manuscript generation.
Hong Kong University Data Science Lab's AI-Researcher, focused on statistical rigor and manuscript quality.
Seven dimension groups applied consistently across all systems.
|
Compute & Runtime
|
Runtime duration
Compute cost
Token consumption
Pipeline completion
|
|
Failure Handling
|
Error recovery
Stability across runs
Missing data degradation
Completion under constraint
|
|
Hypothesis & Ideas
|
Scientific novelty
Research question clarity
Hypothesis originality
Literature alignment
|
|
Experimentation
|
Experimental design
Execution autonomy
Iterative self-correction
Tool integration
|
|
Statistical Rigor
|
Method appropriateness
Statistical validity
Reproducibility
Confounder handling
|
|
Literature & Evidence
|
Search breadth
Citation accuracy
Evidence synthesis
Retrieval quality
|
|
Manuscript Quality
|
Coherence & structure
Writing clarity
Evidence alignment
Peer-review readiness
|