LaunchedEditorial Listing

PaperOrchestra

Google Cloud AI Research · PaperOrchestra: Google Cloud AI Research's Open-Source Multi-Agent Pipeline That Turns Experiment Notes Into a LaTeX Paper

Open PaperOrchestra

PaperOrchestra is an open-source (Apache-2.0) multi-agent pipeline from Google Cloud AI Research that turns an idea summary and an experimental log into a compiled LaTeX manuscript, with an outline, a literature review whose citations are checked against Semantic Scholar, optional generated plots and diagrams, and review-driven revisions. It suits AI and ML researchers who have finished their experiments and want a first full draft in a conference template to edit and verify.

PricingFree
Setupmedium
Runs onSelf-hosted
APINo
Open sourceYes
DocsYes
CategoryResearch
ResearchAcademic PapersPaper WritingLaTeXLiterature ReviewCitationsSemantic ScholarMulti-AgentGeminiOpenAIOpen SourceSelf-HostedPythonCLIStreamlitBYOK

Best for

AI and ML researchers with finished experiments, Gemini or Vertex AI access and a local TeX install, who want a cited first draft in a CVPR- or ICLR-style LaTeX template to revise and fact-check

Not ideal for

Researchers who need experiments designed or run, writers outside AI and ML or targeting non-LaTeX venues, teams limited to Anthropic or local models, anyone wanting a hosted, supported product, and anyone planning to submit output without checking every claim and citation

Who it's for

AI and ML researchers, graduate students and research labs drafting conference papers, and researchers studying automated scientific writing

Capabilities

  • Takes unstructured pre-writing materials (an idea summary, an experimental log and optional figures) and produces a full LaTeX source tree and compiled PDF in a target conference template
  • Outline Agent plans the section structure, figures and literature search directions before any writing starts
  • Two-stage literature agent: Gemini with Google Search grounding discovers candidate papers in parallel, then each candidate is matched against the Semantic Scholar API and filtered by a `--research_cutoff` year-month before it can be cited
  • Optional Plotting Agent (`--use_plotting true`) generates statistical plots and conceptual diagrams with an image model (gemini-3-pro-image-preview by default) and up to three critic rounds per figure by default; without it, you supply figures in a `figures/` folder
  • Section Writing Agent writes the complete manuscript, and a Content Refinement Agent compiles it, runs a simulated peer review, revises, and keeps a revision only if the overall review score rises or ties without a net loss across sub-scores (up to three iterations), plus a visual formatting check on PDF page screenshots
  • Separate CLI model settings for the writer and literature-synthesis agents, the reflection (review) agents, the plotting agent and the image model
  • Local Streamlit demo for uploading an idea and experimental log, choosing a template and one of two Gemini models, and downloading the PDF and LaTeX, with optional emailing of results through your own SMTP account
  • Bundled CVPR 2025 and ICLR 2025 templates with conference guidelines; the README says new venues are added as a template subfolder
  • Ships the autoraters used in the paper: citation F1, literature-review quality, side-by-side literature-review and paper quality, and simulated peer review

Limitations

  • The README states it is not an officially supported Google product; the repository has no tagged releases and its requirements file pins no versions
  • It writes up finished work only: it does not design or run experiments, so the quality of the draft depends on the idea summary and experimental log you provide
  • Model support is limited to Gemini (Vertex AI or a Gemini API key) and OpenAI; model names containing gpt, o1 or o3 go to OpenAI and everything else goes to Gemini, so Anthropic and open-weight models are not supported without code changes
  • Literature discovery is hardcoded to the gemini-3-flash-preview model with Google Search grounding, and the default figure model and the PDF formatting check also call Gemini, so a Gemini or Vertex AI credential is needed even if OpenAI handles the writing
  • The README still says the PaperWritingBench dataset will be released later, although the authors have published it on Hugging Face, so the repository documentation lags behind the project page
  • The pipeline compiles with pdflatex and bibtex, so a local TeX distribution is required, which the README does not mention
  • Only CVPR 2025 and ICLR 2025 templates are bundled, and the authors' evaluation covers AI conference papers only; no results for other fields or formats are reported
  • The authors note that relying on PaperBanana for figures limits their control over figure hallucinations, and the refinement loop optimizes an LLM reviewer's score; there is no interactive, human-in-the-loop steering of drafts
  • No per-paper dollar cost is published; you pay Gemini and OpenAI API usage directly, and the paper reports about 60 to 70 LLM calls per manuscript

Use cases

  • Turning a finished ML project's notes and result tables into a first full CVPR- or ICLR-format draft to edit
  • Drafting an introduction and related-work section whose references are checked against Semantic Scholar and limited to papers before a chosen cutoff date
  • Generating result plots and a method diagram from an experimental log when no figures exist yet
  • Benchmarking automated paper-writing systems with the bundled autoraters and the PaperWritingBench dataset
  • Studying or extending a multi-agent writing pipeline, for example by adding a new venue template or swapping the reviewer model

Our take

PaperOrchestra targets one specific step: turning finished experiments into a draft with a defensible related-work section. Its literature step separates discovery (Gemini with search grounding) from verification (Semantic Scholar matching and a date cutoff), a design aimed at invented or anachronistic citations, and the refinement loop discards revisions that a reviewer model scores lower. Treat it as a research code release rather than a product. It is tied to Gemini in practice, ships two templates, has no releases, and its headline results come from the authors' own benchmark, where human-written papers still came out ahead. Run it for a first draft, then rewrite and check every claim yourself.

Who should use it

ML and AI researchers with completed experiments and a clear write-up of results, who have Gemini or Vertex AI access, are comfortable with Python and LaTeX, and want a cited first draft in a CVPR- or ICLR-style template to revise. It also suits researchers benchmarking or extending automated paper-writing systems.

Who should skip it

Researchers who need help running experiments, people outside AI and ML or targeting non-LaTeX venues, teams restricted to Anthropic or local models, anyone wanting a supported hosted product, and anyone who would submit generated text without independent checking and appropriate AI-use disclosure.

Strengths

  • Apache-2.0 code from Google Cloud AI Research, with the method, benchmark and autoraters published alongside it
  • Works from loose notes and logs rather than a fixed experiment pipeline or a prepared reference list
  • Candidate citations are matched against Semantic Scholar and limited to a research cutoff date
  • Optional plots and diagrams and review-driven revisions are part of the pipeline, ending in a compiled PDF
  • Choice of a CLI for batch runs or a local Streamlit demo

Weaknesses

  • Not an officially supported Google product, with no tagged releases
  • Effectively requires Gemini access even when OpenAI does the writing; no Anthropic or local model support
  • Needs a TeX installation that the README does not mention
  • Only CVPR 2025 and ICLR 2025 templates bundled, and evaluated only on AI papers
  • Quality results come from the authors' own benchmark and reviewer models

PaperOrchestra pricing

Open source

Free

  • Apache-2.0 license
  • Runs on your own machine from a Python CLI or local Streamlit demo
  • Bring your own Gemini or Vertex AI credentials, and optionally an OpenAI key

Free tier limits: No limits from the project. Costs come from Gemini, Vertex AI or OpenAI API usage; a Semantic Scholar API key is optional.

Note: No per-paper dollar cost is published. The authors report about 60 to 70 LLM calls and a mean of 39.6 minutes per manuscript on a 10-paper test with a single worker, using Gemini 3.1 Pro for writing and Gemini 3 Flash with Google Search grounding for literature discovery.

Technical specs

Modalities

Text, Image

Available models

Default writer, reflection and plotting model: gemini-3.1-pro-previewLiterature discovery: gemini-3-flash-preview with Google Search grounding (hardcoded)Default image model for figures: gemini-3-pro-image-previewOpenAI models selectable by name for the writer and reflection agents (names containing gpt, o1 or o3)Gemini via Vertex AI (VERTEX_AI_PROJECT and VERTEX_AI_LOCATION) or GEMINI_API_KEY; Vertex AI takes precedence when both are set

Where PaperOrchestra excels

Drafting a conference submission from a finished ML project

Given an idea summary and an experimental log, the pipeline outlines the paper, writes every section into the CVPR or ICLR template, and compiles a PDF that the authors can then revise.

Building a related-work section with verified citations

Candidate papers found through search-grounded Gemini calls are matched against Semantic Scholar and filtered by a research cutoff, a step designed to keep fabricated or too-recent references out of the draft; the authors still require human checking.

Evaluating automated paper-writing systems

The repository includes the citation F1, literature-review quality, side-by-side and simulated-review autoraters from the paper, and the authors published PaperWritingBench's 200 reverse-engineered paper inputs on Hugging Face.

PaperOrchestra vs. competitors

PaperOrchestra vs. The AI Scientist

The AI Scientist is an end-to-end research system that drives coding agents through experiments, ablations and a sealed evaluation before writing a LaTeX paper. PaperOrchestra only does the writing, starting from your own idea summary and experimental log, with dedicated agents for Semantic Scholar-verified literature review and for figure generation. Its paper compared against AI Scientist-v2's writing, not the current v3.

PaperOrchestra vs. OpenScience

OpenScience is an interactive research agent and workbench that searches literature, runs analysis code and dispatches experiments under your direction. PaperOrchestra is a batch pipeline for one job, converting finished results into a conference-format manuscript, with no code execution or interactive steering.

PaperOrchestra vs. ClawsGO Science

ClawsGO Science is a hosted, credit-based research agent that runs analyses on a cloud machine and can draft cited LaTeX papers. PaperOrchestra is free, open-source code you run yourself with your own Gemini or OpenAI keys, and it is focused on writing up results you already have.

PaperOrchestra vs. Deep Review

Deep Review is a Google Research tool that reviews an existing paper or technical document for logical gaps, errors and missing related work. PaperOrchestra writes the draft in the first place, so the two can be used in sequence: draft with PaperOrchestra, then get a second read from a reviewer such as Deep Review.

Frequently asked questions

What does PaperOrchestra do?

PaperOrchestra turns research materials you already have, mainly a short idea summary and an experimental log, into a full LaTeX paper in a conference template. Specialized agents write an outline, search for and verify related work, optionally generate plots and diagrams, write every section, and revise the draft using a simulated peer review before compiling the PDF.

Is PaperOrchestra open source and free?

Yes. The code is licensed under Apache-2.0 and is free to use. You pay for the model APIs it calls (Gemini or Vertex AI, and optionally OpenAI). The README notes that it is not an officially supported Google product.

Which models and API keys does PaperOrchestra need?

It supports Gemini, through Vertex AI or a Gemini API key, and OpenAI models. The defaults are gemini-3.1-pro-preview for writing and review and gemini-3-pro-image-preview for figures, and literature discovery always uses gemini-3-flash-preview with Google Search grounding, so a Gemini credential is needed in practice. An OpenAI key is only needed if you choose OpenAI models. The README lists a Semantic Scholar API key for citation verification, but the code also sends Semantic Scholar requests without one. Anthropic and local models are not supported without code changes.

How do I install and run PaperOrchestra?

Create a Python 3.11 environment (the README uses Conda), run `pip install -r requirements.txt`, and put your API keys in a `.env` file. Then run `python paper_writing_cli.py` with `--raw_materials_dir`, `--latex_template_dir` (for example `templates/iclr2025`) and optionally `--output_dir` and `--use_plotting true`, or start the Streamlit demo with `streamlit run frontend/app.py`. The code compiles papers with pdflatex and bibtex, so you also need a TeX distribution installed.

Does PaperOrchestra run experiments?

No. It is a writing pipeline. It expects your experiments to be finished and described in an experimental log, and it does not write or execute experiment code. End-to-end systems such as The AI Scientist also run the experiments before writing.

How good are PaperOrchestra's papers?

In the authors' own evaluation on their PaperWritingBench, 11 AI researchers made 180 side-by-side comparisons over 40 sampled papers and preferred PaperOrchestra's manuscripts over a single-agent baseline and AI Scientist-v2 by absolute win-rate margins of 14 to 38 points for overall quality and 50 to 68 points for literature review. The authors also report that a quality gap remains against the original human-written papers. These are the authors' results on their own benchmark, not independent tests.

Can I submit a paper written by PaperOrchestra?

The authors describe it as an assistive drafting tool, not an author, and state that human researchers keep full accountability for the accuracy, originality and validity of every claim in a generated manuscript. No feature for AI-use disclosure or venue-policy checks is documented, so treat the output as a draft, verify every citation and result, and follow your venue's rules on AI assistance.

Is the PaperWritingBench dataset available?

Yes. The authors published PaperWritingBench on Hugging Face (yiwen-song/PaperWritingBench) under Apache-2.0, with raw materials for 200 CVPR 2025 and ICLR 2025 papers, and the project page links to it. The repository README still says the dataset will be released later.

PaperOrchestra vs The AI Scientist: what is the difference?

The AI Scientist from Sakana AI runs the research itself, driving coding agents through experiments and ablations before writing the paper. PaperOrchestra starts after the research is done and focuses on writing: literature search with citation verification, figures, section writing and review-driven revision from your notes and logs. The PaperOrchestra paper compared against AI Scientist-v2's writing stage, not the current v3.

Integrations & fit

Gemini APIVertex AIOpenAI APIGoogle Search groundingSemantic Scholar APILaTeX (pdflatex, BibTeX)StreamlitSMTP email
Good fit forSolo / individual, Startup / small team
Pricing modelFree· No cost to start
See pricing on PaperOrchestra →

Alternatives to consider

About PaperOrchestra

PaperOrchestra is the code release of a research paper by Yiwen Song, Yale Song, Tomas Pfister and Jinsung Yoon of Google Cloud AI Research, first posted on arXiv in April 2026 and published in the google-research GitHub organization under Apache-2.0. It is a writing pipeline, not an experiment runner. You give it a folder of raw materials (by default `idea_sparse.md`, a short idea summary, and `experimental_log.md`, your results and observations), a LaTeX template directory with conference guidelines, and an output folder. Five specialized agents then take over, with plotting and literature search running in parallel. An Outline Agent plans the paper, the figures and the literature searches. A Plotting Agent, when `--use_plotting` is on, generates statistical plots and conceptual diagrams with a critic loop (otherwise you supply your own figures). A Literature Review Agent runs Gemini with Google Search grounding to find candidate papers, then matches each one against the Semantic Scholar API and drops anything published after a research cutoff (by default the current month, set with `--research_cutoff`). A Section Writing Agent writes the full LaTeX manuscript, and a Content Refinement Agent compiles it, scores it with a simulated peer review (AgentReview in the paper), and keeps a revision only when the overall score rises (or ties without a net loss across sub-scores), for up to three rounds. It runs from a Python CLI or a local Streamlit demo, and ships CVPR 2025 and ICLR 2025 templates plus the autoraters used in the paper. Models come from Gemini (through Vertex AI or a Gemini API key) or OpenAI. You can point the writer and reviewer at OpenAI models, but literature discovery always calls a Gemini model, so a Gemini credential is needed in practice. In the authors' human evaluation on their own PaperWritingBench (built from 200 CVPR and ICLR 2025 papers), 11 AI researchers compared manuscripts for 40 sampled papers and preferred PaperOrchestra over a single-agent baseline and AI Scientist-v2 by absolute win-rate margins of 14 to 38 points for overall quality and 50 to 68 points for literature review, while human-written papers still came out ahead. The tradeoffs: the README calls it not an officially supported Google product, there are no tagged releases, only two templates are bundled, you pay your own API costs (the paper reports about 60 to 70 LLM calls and a mean of 39.6 minutes per paper), and the authors state that humans remain accountable for every claim and citation in the output.

Updates from PaperOrchestra

New FeatureProvider keys made optional and dependency fixes merged

Three community pull requests were merged: API keys are now required only for the LLM provider a model actually uses (literature discovery still calls Gemini), missing packages were added to requirements.txt, and invalid escape sequences in LaTeX prompt strings were fixed.

LaunchPaperWritingBench dataset published on Hugging Face

The authors published PaperWritingBench on Hugging Face under Apache-2.0: reverse-engineered raw materials (idea summaries and experimental logs) for 200 CVPR 2025 and ICLR 2025 papers, used to evaluate the pipeline. The project page links to it.

LaunchPaperOrchestra code published

The initial code commit in the google-research GitHub organization added the Apache-2.0 PaperOrchestra code, with the CLI, Streamlit demo, CVPR and ICLR templates and evaluation autoraters.

LaunchPaperOrchestra paper posted on arXiv

The paper "PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing" (arXiv:2604.05018) introduced the framework and the PaperWritingBench benchmark.

Are you the founder? Claim this listing →