The Coworker benchmark shows you, on your own data, how Claude performs with Coworker compared with Claude on its own. Setup takes 30 to 45 minutes. Do it before your setup call, so the call can start the run instead of installing tools.
Note: The benchmark is through invitation from the Coworker team. Reach out to your Coworker point of contact first to get started
What is the benchmark
The Coworker Benchmark is a Mac app that runs on your computer. It asks Claude Code the same set of questions twice:
Claude: Claude Code with your own Claude connectors (such as Slack, Google Drive, Gmail and Jira) and no Coworker.
Coworker: Claude Code with only the Coworker MCP and no other connectors.
Both sides use the same model (Sonnet 5.5) at the same effort level (High). Each side runs in its own session with no shared memory, so neither side helps the other. After the run, you compare each pair of answers without knowing which side wrote which. Coworker then shares cost, speed and quality results with you.
Before you start
You need:
A Mac. The benchmark app is macOS only.
Claude Code in the terminal. The benchmark cannot run from Claude on the web, the Claude desktop chat app or Cowork. Claude Code inside the Claude desktop app is not enough. You need the
claudecommand in Terminal.A Claude plan with enough usage. The run uses your own Claude account. Tell your Coworker contact whether you are on a Team or Enterprise plan, and whether you use a subscription or an API key.
A Coworker account in your company's Coworker workspace.
The tools you use every day. The benchmark works best for people who use the tools being tested.
How long it takes
Step | Who | Time |
Prep (steps 1 to 6 below) | You, with help from Coworker if needed | 30 to 45 minutes |
Setup call | You, Coworker Customer Success and a Coworker engineer | 30 minutes |
Benchmark run | Your Mac, monitored by Coworker | Usually under an hour for 60 questions. Can take longer, so start it near the end of your day. |
Coworker check | Coworker | Before you see results |
Quality Review | You | 60 to 90 minutes for 60 questions |
Results and debrief | You and Coworker | A meeting scheduled after results are shared |
Step 1: Install Claude Code in your terminal
Skip this step if typing claude in Terminal already opens Claude Code.
Open Terminal. Press Command + Space, type Terminal and press Return.
Paste this command and press Return. It runs Anthropic's official installer:
curl -fsSL https://claude.ai/install.sh | bashOpen a new Terminal window. Type
claudeand press Return.Sign in with the same Claude account you normally use.
The full guide is the Claude Code quickstart.
Already have Claude Code? Keep it up to date. The Coworker plugin needs Claude Code version 2.1.224 or later. Type claude --version in Terminal to check, and run claude update to update.
Good to know: macOS may ask whether Terminal can access folders such as Desktop or Documents. This is a normal macOS permission prompt for Claude Code.
Never used a terminal? That's fine. Tell your Coworker contact before the call and we will walk you through it. The benchmark app can also install Claude Code for you: when it opens and can't find Claude Code, click Install Claude Code.
Step 2: Connect your data sources in Coworker App
Coworker needs your own connections to the tools you use, so your answers include what you personally can see.
Open Data Sources in Coworker. You can also click Data Sources at the top right of the Work page, or click your name at the bottom left and choose Data Sources.
Open the Available tab and click Connect on each tool you use every day. Common ones are Slack, Google Drive, Gmail, Google Calendar, Jira, Confluence, HubSpot, Salesforce, GitHub, Notion and Linear.
Finish each sign-in. The source moves to the Connected tab.
Things to know:
Connect each source yourself, even if your admin already connected it for the company. Sources on the Org-wide tab were connected by an admin. The benchmark compares your personal connections, so connect your own account too.
"Admin needed" means an admin must connect that source for your workspace before you can connect yours. Some sources, such as Jira, Confluence and Salesforce, need this. Click Request to Connect to ask your admin.
"Blocked" means an admin has blocked that source for your workspace.
Slack on Enterprise Grid: some Enterprise Grid policies block installing the Coworker Slack app. If you're on Enterprise Grid, tell your Coworker contact before the call.
Connect early. Coworker needs time to bring in your data after you connect, so finish this step as soon as you can, not on the day of the call.
Step 3: Connect the same sources in Claude
The comparison is only fair if Claude has the same tools as Coworker. For every source you connected in Step 2, connect that same tool to Claude.
In Claude on the web or the desktop app, go to Customize → Connectors.
Add and connect each tool you connected in Coworker, for example Slack, Google Drive, Gmail, Google Calendar and Atlassian (Jira and Confluence). Connect them one at a time and finish each sign-in.
Connectors you add in Claude also appear in Claude Code. To check, run
claudein Terminal and type/mcp. Each connector is listed with "claude.ai" in front of its name. If a connector says it needs authentication, select it and sign in.
Team or Enterprise plan? An owner of your Claude organization may need to add a connector before you can connect it.
A tool that isn't in Claude's connector list? You can add it as an MCP server in Claude Code. See Connect Claude Code to tools via MCP. Add it for your user (user scope), not only for one project folder, so the benchmark app can see it.
You don't need a perfect match. The benchmark app shows which sources are connected on each side and only asks questions about sources you turn on. The more sources match, the more complete your comparison.
Step 4: Install the Coworker MCP in Claude Code
First, check whether you already have it. Your company may have installed it for everyone. In Terminal, run claude, then type /mcp. If you see an entry with coworker in its name and it shows as connected, skip to Step 5.
If it's missing:
Open the Coworker MCP setup page for Claude Code. It shows the exact commands for your workspace.
In Claude Code, run this command:
/plugin marketplace add https://app.coworker.ai/plugin/marketplace.jsonThen run this command:
/plugin install coworker@coworker. If Claude Code asks where to install it, choose user scope (just for you).Sign in. Type
/mcp, select coworker then authenticate and complete the sign-in in your browser. You can also ask Claude any question about your company and a sign-in window opens. You only need to sign in once.Check that it worked: type
/mcpagain. Coworker should show as connected.
Getting an admin approval error? On some Claude Team and Enterprise plans, an admin must approve the connector. Ask your Claude admin to approve it, or to install Coworker for your whole organization (see "For admins" below).
Step 5: Install the Coworker Benchmark app and sign in
Your Coworker contact will send you the Coworker Benchmark app as a .dmg file. Open it and drag the app into Applications.
Open the app and click Sign in with your browser.
Your browser opens the Coworker sign-in page. Sign in with Google or your email and password.
The app shows an 8-character code. Enter it in your browser and click Continue. When you see You're signed in, go back to the app.
If you belong to more than one Coworker workspace, choose the one your benchmark was set up in.
If the app says Setup is in progress, Coworker is still preparing your benchmark. Click Check again later, or ask your Coworker contact.
Step 6: Share your own questions (optional)
By default, the benchmark uses a standard question library. The app only runs questions that match the sources you turn on.
You can also use your own questions:
Templated questions: standard questions filled in with your own names, such as real customers, projects or teams. You fill these in the app under Your variables.
Your own questions: a list of questions your team really asks. Send it to your Coworker contact at least 48 hours before the run. Coworker checks that each question can be answered from your connected sources.
Many teams start with the standard questions and then considering adding some of their own for a second run.
Pre-call checklist
Claude Code opens when you type
claudein Terminal, is up to date and is signed inYour data sources are connected in Coworker under your own account
The same sources are connected in Claude and show in
/mcpCoworker shows as connected in
/mcpThe Coworker Benchmark app is installed and you are signed in
Your own questions (if any) were sent at least 48 hours ago
You know whether your laptop goes to sleep on its own
Time is blocked for the 30-minute call and the later review
For admins: set things up for your whole company
These steps save every tester from doing them one by one.
Install Coworker for your Claude organization. The Coworker MCP page has an Install for your Claude org tab with steps for Claude Team and Enterprise. This also avoids admin approval errors for members.
Connect sources for your workspace. Open Admin Source Settings and click Connect for the whole workspace on each source. Sources such as Jira, Confluence and Salesforce need this before members can connect their own accounts.
Testers still connect their own accounts. The benchmark compares each tester's personal connections, so ask every tester to complete Step 2.
The setup call (30 minutes)
On the call, we will:
Open the benchmark app and check your sources on the Data sources page.
Review the question list on the Questions page.
Agree on when to start the run. Please post in your shared Slack channel with Coworker when you start, so an engineer can watch for problems.
We will record the call and share the recording with you.
Running the benchmark
The app has five tabs: Data sources, Questions, Run, Quality Review and Results.
Data sources
This page lists each source and whether it is connected In Coworker and In Claude:
✓ Connected.
⚠ Only one side has it. This gap can skew the benchmark. Click the ⚠ to see how to connect the missing side.
✕ Neither side has it. It is off by default.
After you connect something, click Re-check connectors. Use the Enabled switch to turn sources on or off. Both only turns on just the sources connected on both sides. Click Next → when you're done. The app then checks that the Coworker MCP is connected before you can continue.
Questions
This page shows the questions that will run, for example "Your benchmark questions (55 of 60 will run)". You don't need to answer or type anything here. It's a preview. Turn off any question that doesn't make sense for you. Questions that need a source you turned off are hidden automatically.
Run
Click Start benchmark. Every question runs on both sides, so 60 questions become 120 answers. The app runs several questions at once.
While it runs:
Keep your laptop open and plugged in. If it sleeps, the run pauses. Closing the lid will put most Macs to sleep. Some company laptops sleep on a timer, so check in midway.
If the app closes or your Mac sleeps, nothing is lost. Reopen the app and the run resumes. If it doesn't, click Resume run, or contact your Coworker contact.
You don't need to approve anything. Claude Code runs without asking for permission on each step.
If Claude needs you to sign in again, the run pauses with a message. Run
claudein Terminal, sign in, then click Resume in the app.If Coworker needs reconnecting, the app shows Coworker MCP needs to be reconnected. Reconnect it in
/mcpin Claude Code, then click Retry now.
Claude usage and cost
The run uses your Claude account. A full run of 60 questions usually uses about 40% to 50% of a Claude session's usage, so start it near the end of your day if you use Claude for other work.
Every question runs at High effort, which uses a little more than a default Claude Code session.
Ask your Coworker contact about covering the cost of the benchmark.
If you hit your Claude usage limit, the run waits and resumes by itself when the limit resets.
Don't want to wait? Under Claude API key on the Run page, paste an Anthropic API key (it starts with
sk-ant-) and choose Fall back on limit. Relevant API key usage costs still apply. The key must come from a different Anthropic account than your Claude subscription. It is stored encrypted on your Mac and is never sent to Coworker.
Reviewing the answers (Quality Review)
After the run, Coworker checks the results first. The Quality Review tab opens when Coworker unlocks it.
For each question, you see Answer A and Answer B side by side. You don't know which came from Coworker and which came from Claude alone, and costs and timings are hidden.
Choose Prefer A, Prefer B, Tie or Skip. Keyboard shortcuts: A, B, T and S.
Rate each answer as Accurate, Partly right or Inaccurate. This is required for every pair you don't skip.
When you're done, click Submit review, then Yes, submit. You can't change your picks after you submit.
What to judge: overall quality, not expert-level checking. Does the answer seem right? Is it easy to read and to act on? You don't need to be an expert on every question.
Tip: 60 pairs take about 60 to 90 minutes. Split them across teammates if you're short on time. Download CSV exports the pairs.
Your results
When Coworker has verified the results, the Results tab opens. It shows:
Quality: how often you preferred the Coworker answer
Cost: estimated cost per question for each side
Speed: time to answer each question
Failures: where one side or both failed to answer
Question by question: tokens, tool calls, cost and time for each answer
Cost and speed are compared only on questions where you rated both answers accurate, so the accuracy ratings matter.
Coworker will schedule a debrief to go through the results with you. If something went wrong with the setup, Coworker can allow one more run. Keep the app installed until your Coworker contact confirms you won't need it again.
Troubleshooting
What you see | What to do |
The app can't find Claude Code | Install it from the app, or follow Step 1. Then open a new Terminal window and check that |
"Coworker MCP not connected" when you click Next | Run |
⚠ on the "In Claude" side | Add that connector in Claude (Step 3), authenticate it in |
⚠ on the "In Coworker" side | Click Connect in Coworker in the pop-up, connect your own account, then click Re-check connectors. |
"Not individually connectable in Coworker" | That source can't be connected per person. Ask your Coworker admin. |
An admin approval error when installing Coworker in Claude | Ask your Claude admin to approve the connector or install it for the whole organization. |
The app looks out of date | Press Command + R to reload it. |
The run stopped after your Mac slept | Reopen the app. It resumes, or click Resume run. |
"Claude usage limit reached" | The run resumes when your limit resets, or add an API key (see "Claude usage and cost"). |
"Setup is in progress" | Coworker is still preparing your benchmark. Click Check again later. |
FAQ
Do I need Claude Code to run the benchmark?
Yes. It runs only from Claude Code in the terminal, not from Claude on the web, the Claude desktop chat or Cowork.
I use Claude Code inside the Claude desktop app. Is that enough?
No. You need Terminal version that starts when typing claude in your Terminal. See Step 1.
Does the benchmark work on Windows?
No. The benchmark app is for macOS only.
I've never used a terminal. Can I still take part?
Yes. Tell your Coworker contact and we'll walk you through setup.
How long does all of this take?
About 30 to 45 minutes of prep, a 30-minute setup call, the run itself (mostly unattended), and 60 to 90 minutes to review 60 questions.
Which data sources should I connect?
The tools you use every day, such as Slack, Jira, Google Calendar, Google Drive, Gmail and your CRM. Connect them yourself so your own data is included.
What does "parity" mean?
The same tools connected both in Coworker and in Claude, so the comparison is fair. The app marks each source ✓, ⚠ or ✕.
Do I have to connect every source?
No. Questions that need a source you turn off are skipped automatically. More matching sources give a more complete comparison.
My admin installed Coworker for the company. Do I still need to install it?
No. Check that it appears in /mcp in Claude Code and is connected.
Does the Claude-only side use Coworker?
No. The app removes Coworker from the Claude-only side automatically. You don't need to turn anything off.
Will this use my Claude credits?
Yes. A full run typically uses about 40% to 50% of a Claude session's usage based on Team plan with default limits. Ask your Coworker contact about the cost coverage.
Is my API key shared with Coworker?
No. It is stored encrypted on your Mac and never leaves it.
What if my laptop sleeps or the app closes?
The run pauses and resumes when you reopen the app.
Do I need to answer the questions shown in the app?
No. The Questions page is a preview. Turn off any question you'd rather skip.
Can I use my own questions?
Yes. Send them to your Coworker contact at least 48 hours before the run. See Step 6.
Will I know which answer came from Coworker?
No. The review is blind so your preference is unbiased.
What happens after I submit my review?
Coworker verifies the results, opens your Results tab and schedules a debrief.
Do I need to uninstall the app afterwards?
No. Keep it until your Coworker contact confirms you don't need another run.