Controlled comparison · measurement report

Appwrite MCP vs Supabase MCP: a 14-task delegation benchmark

Both servers were given an identical schema, identical deterministic seed data, and the same fourteen jobs a developer would realistically hand to an MCP server. Every call is logged. No composite score is computed — the six axes are reported separately and left unweighted.

1Method and environment

Fresh, empty projects were created on both platforms specifically for this run. Regions were paired to Frankfurt on both sides to limit the network confound. The suite was executed once in each direction: Pass 1 with Appwrite leading, Pass 2 with Supabase leading, in fresh namespaces.

	Appwrite	Supabase
Project	`mcp-bench-appwrite`	`hpbrqwobtpyrqeguawcn`
Region	fra (Frankfurt)	eu-central-1 (Frankfurt)
Backend version	Appwrite Cloud 1.9.6	Postgres 17.6.1.063
Plan	official-tier-1	free ($0/mo confirmed via `get_cost`)
MCP auth mode	Hosted HTTP, OAuth console session	Hosted, service-role credentials
Tool exposure	Hidden catalog; `search_tools` → `call_tool`	~29 tools exposed directly
Pass 1 namespace	database `bench`	schema `public`
Pass 2 namespace	database `bench2`	schema `bench2`

Run date 2026-08-10, 11:06–11:22 UTC. Client: macOS 25.5.0, arm64, single machine, single session.

Schema and seed

Two tables. authors: email (unique), name, joined (datetime), active (boolean). books: title, rating (float), published (boolean), created (datetime), plus a many-to-one link to authors. Seed: 5 authors and 50 books generated from one deterministic script so both sides received byte-identical values.

One amendment to the approved suite. T7 was specified as published = true AND rating >= 4.0. Only 7 of 50 rows matched, so limit 10 offset 10 would have returned an empty page and tested nothing. The threshold was lowered to rating >= 2.0 (20 matching rows) before any data was inserted. The expected page was computed in advance — b25, b20, b19, b14, b13, b11, b10, b07, b05, b04 — and both servers returned exactly that, in that order, in both passes.

2Per-task results

“Calls” counts round-trips through the MCP channel to complete the task, including failed attempts and any verification call needed to confirm the result. Where a first attempt failed because of an inaccurate tool description, the Pass 2 figure shows the cost once the correct shape was known.

#	Task	Appwrite	Calls P1 / P2	Supabase	Calls P1 / P2
T1	Create empty project, confirm zero state	Pass (bug) Create returned a Pydantic parse error; the project was created. False negative.	3 / —	Pass `ACTIVE_HEALTHY` immediately; cost gate first.	5 / —
T2	Table, 4 typed columns, unique constraint	Pass First try failed: description documents `columns`, API requires `attributes`.	4 / 1	Pass One `apply_migration`.	1 / 1
T3	Second table, 5 typed columns incl. float	Pass First try failed: description lists `float`, API requires `double`.	2 / 1	Pass	1 / 1
T4	Relationship / foreign key across tables	Pass Correct first try despite undocumented enums.	1 / 1	Pass `ALTER TABLE … ADD CONSTRAINT`.	1 / 1
T5	Insert 5 rows, one per call	Pass	5 / 1*	Pass	5 / 1*
T6	Bulk-insert 50 rows	Pass (conditional) Bulk rejected when the table has a relationship column → 50 individual calls. With bulk before the relationship: 1 call.	51 / 1	Pass Single multi-row `INSERT`.	1 / 1*
T7	Filtered + sorted + paginated read	Partial Correct 10 IDs, correct order, correct `total: 20` — but no column values returned. 10 extra `get_row` calls needed for the data.	1 (+10) / 1	Pass Rows returned with values.	1 / 1
T8	Update a row, verify new value	Pass Response echoes updated row.	1 / —	Pass `UPDATE … RETURNING`.	1 / —
T9	Delete a row, verify absence	Pass Delete returns a 0-byte binary blob written to disk rather than a success value.	2 / —	Pass	2 / —
T10	Join read: titles with author names	Pass (per row) `select: ["title","author.name"]` traverses correctly, but only via `get_row` — 1 call per book.	N / —	Pass One SQL join, all rows.	1 / —
T11	Apply a permission / RLS rule	Pass `read("users")` table ACL.	1 / —	Pass Enable RLS + authenticated-only policy.	1 / —
T12	Verify the rule blocks an unauthorized read	Pass Guest → `401 user_unauthorized`. Control table also denied.	OOB	Pass Anon → `200 []`, no rows leaked. Control table was readable.	OOB
T13	File upload and retrieval	Pass Bucket → upload → retrieve; byte-identical round trip. Binary lands as a local file, not inline JSON.	3 / —	Not supported via MCP No Storage tools exist on this server. Supabase Storage exists as a product (REST/SDK/CLI).	—
T14	Three deliberate invalid requests	Partial Unknown column and bad type: precise. Unique violation: misreported as a row-ID collision.	3 (+2) / 1	Pass All three return SQLSTATE + DETAIL + caret position.	3 / 1

* In Pass 2 Supabase collapsed T5, T6 and T4 into a single statement (55 rows plus the foreign key). Appwrite Pass 2 collapsed T5 into one bulk call. “OOB” = verified out of band; see §3. “N” = one call per row returned.

Two findings that needed a control to state fairly

Appwrite's bulk-insert limit is an ordering effect, not a missing capability. In Pass 1, tables_db_create_rows rejected all 50 rows: “Bulk create is not supported for tablesDB with relationship columns.” The fallback cost 50 round-trips. In Pass 2 the same 50 rows went in as one call — because the bulk ran before the relationship column was added, and the relationship was added afterwards without error. So T6 costs 1 call or 50 depending purely on the order operations are issued. That is worth knowing and is not documented in the tool descriptions.

The missing column data in T7 is an MCP-layer bug, not an Appwrite API limitation. Because this is the single most consequential defect found, it was attributed directly: a scoped ephemeral key was created and the same query issued against the REST API. REST returned complete rows — title, rating, published, created, author. The MCP server strips them. get_row through MCP returns data; list_rows through MCP does not. The product is fine; the server's response serialization is not.

Appwrite misreports unique-constraint violations. Inserting a duplicate email under a brand-new row ID dup3 returned 409 row_already_exists: Row with the requested ID 'dup3' already exists. Try again with a different ID. The ID was new; the email was the duplicate. Control: the same new-ID pattern with a unique email (dup4) succeeded. Following the error's own remediation advice would not fix the problem.

3The permission test, and why it left the MCP channel

Neither server can honestly test its own access control from the inside: the Supabase MCP server holds service-role credentials, which RLS deliberately does not apply to, and the Appwrite connection is an authenticated console session. Both would have reported “read succeeded” regardless of the rule. T12 was therefore run out of band over plain HTTP, with a control table on each side to distinguish “blocked” from “broken”.

Request	Status	Result
Supabase anon → `books` (RLS on, authenticated-only)	200	Blocked — `[]`, no rows leaked
Supabase anon → `authors` (control, no RLS)	200	Exposed — returned real emails
Appwrite guest → `books` (ACL `read("users")`)	401	Blocked — `user_unauthorized`
Appwrite guest → `authors` (control, default perms)	401	Blocked — deny by default

Both rules did what they were asked to do, so T12 passes on both. The controls expose a genuine difference in default posture: an Appwrite table created through MCP is unreadable until permission is granted, whereas a Supabase table created through MCP is world-readable until RLS is explicitly enabled. Supabase's server does, however, ship the counterweight — get_advisors flagged the exact table at ERROR level with a remediation link, unprompted:

rls_disabled_in_public — ERROR — EXTERNAL
Table `public.authors` is public, but RLS has not been enabled.
https://supabase.com/docs/guides/database/database-linter?lint=0013_rls_disabled_in_public

Appwrite has no equivalent advisory tool on its MCP surface. Two defensible designs: Appwrite prevents the mistake, Supabase permits it and then detects it. Neither is scored as a win on its own.

4The six axes

Scored 1–5, independently, with no composite. Weighting these against each other is a judgement about what you are building, not a property of the measurement.

1 · Tool coverage — fraction of the suite completed unassisted

Appwrite 4/5

13 of 14 fully; T7 returns IDs without values, so the paginated-read task cannot be completed from one call. Every other task including file upload had a working tool path.

Supabase 4/5

13 of 14 fully; T13 has no tool surface at all — the server exposes no Storage operations, so file upload is impossible through MCP regardless of skill.

2 · Schema and description quality — could the tool be used correctly first try?

Appwrite 2/5

Four separate first-try failures traceable to the descriptions: indexes documents a columns field but requires attributes; the column-type list advertises float but requires double; type and on_delete on relationships list no allowed values; file is typed string but requires an object; the queries format is never specified. Pass 2 took 1 call per table once the true shapes were known — the capability was always there, the documentation was wrong.

Supabase 5/5

Every call correct first try. The surface is small and the payload is SQL, so there is almost no schema to get wrong. The one malformed call in the log was an omitted required argument of mine, and it returned a machine-readable Zod error naming the missing path.

3 · Round-trips per task — fewer is better

Appwrite 2/5

Three structural multipliers. Tools are hidden behind search_tools, so each new operation costs a discovery call first (9 across the run, one of which returned five irrelevant tools at score 8 rather than reporting no match). T6 cost 51 calls in the natural task order. T7 needs 11 calls to get what Supabase returns in 1, and T10 costs one call per row.

Supabase 5/5

One call for nearly every task. In Pass 2 the entire seed — 5 authors, 50 books, and the foreign key — went in as a single statement with a verification count attached. Tools are listed directly, so there is no discovery tax.

4 · Error message usefulness — does a failure tell you how to fix it?

Appwrite 3/5

Typed, specific, and genuinely actionable in most cases — row_invalid_structure: Unknown attribute "nonexistent_col", Attribute "rating" has invalid type. Value must be a valid float, and the index error that named the exact missing field. Marked down for two real failures: the unique-violation message that misdiagnoses the cause and gives remediation that does not work, and the project-creation call that reported a Pydantic dict_type error for an operation that had in fact succeeded — the worst failure mode available, since it invites a duplicate-creating retry.

Supabase 5/5

Raw Postgres diagnostics passed through intact: SQLSTATE code, constraint name, offending value, and a caret pointing at the exact token. 23505 … Key (email)=(ada@example.com) already exists requires no interpretation.

5 · Auth and setup friction — steps from zero to first successful call

Appwrite 3/5

get_context is a genuinely good front door — one call enumerated every project, region, and per-service resource count. Against that: project creation returned an error for a success, every mutation needs confirm_write: true, and every project-scoped call needs project_id threaded manually.

Supabase 4/5

Session already authenticated; the new project reported ACTIVE_HEALTHY on creation with no polling. Creating it required a deliberate three-step gate — get_cost → confirm_cost → create_project. That is friction, but it is spend protection working as designed, so it is not scored as a defect.

6 · Latency — dominated by network and client overhead

Read this axis last and weight it near zero. Two things must be stated before the numbers. First, per-call MCP durations are not exposed by the transport, and timing them from inside the agent loop would have measured model inference time rather than the call — so those were not fabricated, and the MCP layer itself is untested for latency. What is reported below is raw HTTPS latency from one laptop to each backend's REST endpoint, measured with curl, 12 samples each. Second, that measures one machine's network path to two Frankfurt endpoints on one afternoon. It is not a statement about either backend's performance.

Endpoint	n	Median	p25	p75	Min	Max
Appwrite (fra)	12	309 ms	291	379	278	441
Supabase (eu-central-1)	12	383 ms	350	721	338	986

Appwrite 4/5

Tighter spread, no cold-start outliers.

Supabase 4/5

Higher median and a much wider p75, but the first three samples hit a freshly provisioned project and look like warm-up; the trailing nine cluster near 350 ms.

The gap is not meaningful at n=12 on a single client. Both are scored 4.

5Where each server is the better choice

Choose Supabase MCP when

The work is schema and data — migrations, bulk loads, analytical reads. One statement does what takes Appwrite many calls, and the gap widens with row count.
You are running an agent on a token or round-trip budget. Fewer calls, no discovery tax, and terser responses.
You need failures to be diagnosable without a human. Postgres error output is the best debugging signal in this comparison by a clear margin.
You want the security posture audited. get_advisors catches misconfigurations that Appwrite's surface has no tool to detect.
Whoever is driving knows SQL. The surface is essentially “a SQL endpoint plus project management” — expressive if you can write it, unhelpful if you cannot.

Choose Appwrite MCP when

The job touches files. Storage works end to end through MCP with a verified byte-identical round trip; Supabase's MCP server cannot do it at all.
The job touches anything beyond the database. Users, teams, functions, sites, messaging and project configuration are all on the MCP surface. Supabase's server covers Postgres, edge functions, branches and logs.
You want deny-by-default access control. A table created through Appwrite MCP is unreadable until you grant permission — the failure mode is a broken read, not a data leak.
You want permissions without writing policy SQL. Declarative ACL strings like read("users") are a lower-ceiling, lower-floor mechanism than RLS.
You need orientation over an unfamiliar account. get_context is the single best discovery call either server offers.

The most actionable finding for Appwrite is cheap to fix. Three of the four axes where Appwrite scores low are documentation and serialization defects in the MCP server, not product gaps: correcting attributes/columns and float/double in the descriptions, documenting the relationship enums and query format, and stopping list_rows from stripping the data field the REST API already returns. The last one alone moves T7 from partial to pass and removes 10 round-trips. None of it requires a change to Appwrite itself.

6Threats to validity

The requester works at Appwrite. This was disclosed up front with an explicit instruction not to let it shade the result. It is nonetheless the largest structural threat here, and it runs in both directions — the correction for a known bias can overshoot. The mitigations applied were: pre-registering the task suite before any tool was called, computing T7's expected result before inserting data, and running a control for every negative finding against Appwrite (the dup3/dup4 pair, the guest-read control table, and the REST attribution test that moved the T7 defect off the product and onto the MCP server). Readers who want to discount for bias should discount the prose, not the call log in §7.
Sample size is one run per task. T2, T3, T4, T6, T7 and T14 were executed twice; T1, T8–T13 were executed once. No task was repeated enough to characterise variance. Every number here is an observation, not an estimate.
Order effects are real and one was quantified. T6 differs by 50× between passes purely on operation order. Pass 2 also benefits from knowledge acquired in Pass 1 — Appwrite's Pass 2 schema calls succeed first try precisely because Pass 1 discovered the correct field names. Pass 1 is the honest cold-start number; Pass 2 is the informed-user number. Both are reported rather than averaged.
Calls were dispatched in parallel where independent. This does not affect round-trip counts, which are the headline metric, but it means wall-clock sequencing in the log is not strictly serial.
Latency was not measured through MCP. Stated plainly in §4: the transport does not expose per-call timings and the agent loop cannot isolate them. The MCP layer's own overhead is untested. The reported figures are out-of-band REST measurements and carry all the caveats listed there.
Only one auth mode per server was exercised. Appwrite ran through a hosted OAuth console session; an API-key or self-hosted deployment may serialize responses or enforce permissions differently. Supabase ran with service-role credentials, which is why T12 had to leave the channel.
The two permission models are not equivalent. Appwrite ACLs and Postgres RLS restrict reads by different mechanisms with different expressive ceilings. T11 and T12 test “did the developer get a working restriction, and at what cost”, not which model is better. RLS can express row-level predicates that a table-level ACL cannot; that capability was not exercised.
Regions were paired but the client was not neutral. Both projects sit in Frankfurt, measured from one machine on one network on one afternoon. Postgres 17 vs Appwrite Cloud 1.9.6 are also simply different systems; nothing here isolates backend performance and nothing should be read as doing so.
Assumptions made without confirmation. That the "fewest calls a competent user would make" is the fair unit for round-trip counting, so Appwrite's inline columns/indexes path was used rather than one call per column. That a failed attempt counts against the task that provoked it. That search_tools calls count as round-trips, since an agent genuinely cannot call a hidden tool without one. Counting them differently would narrow axis 3 but not reverse it.
Residue. Both projects still exist and hold benchmark data. Appwrite carries an extra bench2 database and a bucket; Supabase carries a bench2 schema. The Appwrite authors table burned two ID sequence values on the failed duplicate inserts. A short-lived ephemeral read key was minted for the attribution test and expires on its own.

7Appendix: call log

Chronological, both servers, both passes. Arguments abbreviated where long; verbatim error strings preserved. Wall-clock per call is not listed for the reason given in §4.

Appwrite — Pass 1

appwrite_get_context {}                                    → 5 projects, account, 2 orgs
search_tools "create project"                              → organization_create_project ranked 5th of 8
organization_create_project {mcp-bench-appwrite, fra}      → ERROR "Unable to parse response into Project:
                                                              1 validation error for Project onboarding
                                                              Input should be a valid dictionary"
                                                              [project WAS created — false negative]
appwrite_get_context {project_id: mcp-bench-appwrite}      → exists, all services 0 (fresh)
search_tools "create database/table/index"                 → catalog
search_tools "create a new database"                       → tables_db_create
tables_db_create {bench}                                   → OK status ready
tables_db_create_table {authors, columns[4], indexes:      → ERROR 400 general_argument_invalid
  [{key,type,columns}]}                                       "Index at position 0 is missing required
                                                               field 'attributes' (must be an array)"
tables_db_create_table {authors, indexes:[{attributes}]}   → OK  (response: columns[] indexes[] — stale)
tables_db_list_columns {authors}                           → 4 columns, all status "available"
tables_db_list_indexes {authors}                           → email_unique, type unique, available
tables_db_create_table {books, rating type "float"}        → ERROR 400 "Invalid type for attribute
                                                               'rating': float"
tables_db_create_table {books, rating type "double"}       → OK
search_tools "relationship column"                         → tables_db_create_relationship_column
tables_db_create_relationship_column {books→authors,       → OK status processing  (correct first try)
  manyToOne, cascade}
tables_db_create_row × 5  {authors a1..a5}                 → OK ×5
search_tools "transaction / begin transaction"             → 2nd query returned 5 irrelevant tools @ score 8
tables_db_create_rows {books, 50 rows}                     → ERROR 400 general_bad_request
                                                               "Bulk create is not supported for tablesDB
                                                                with relationship columns"
tables_db_create_row × 50 {books b01..b50}                 → OK ×50  (each response embeds full author obj)
tables_db_list_rows {published=true, rating>=2.0,          → total 20; 10 rows; IDs and order CORRECT;
  orderDesc created, limit 10, offset 10, select[...]}        NO column values returned
tables_db_list_rows {same, no select, limit 3}             → still no column values
tables_db_get_row {b25}                                    → data present  ⇒ list_rows-specific
tables_db_update_row {b25, rating 4.95}                    → OK, echoes updated row
tables_db_delete_row {b50}                                 → 0-byte binary blob written to disk
tables_db_get_row {b50}                                    → ERROR 404 row_not_found (clear)
tables_db_get_row {b01, select["title","author.name"]}     → OK — relationship traversal works
tables_db_update_table {books, permissions:["read(users)"]}→ OK (response includes full columns)
search_tools "storage bucket / upload"                     → storage_create_bucket, storage_create_file
storage_create_bucket {bench-bucket}                       → OK
storage_create_file {bench.txt, base64, 31 B}              → OK sizeOriginal 31, chunks 1/1
storage_get_file_view {bench-file}                         → 31-byte blob → cmp: byte-identical
tables_db_create_row {authors dup1, email ada@…}           → ERROR 409 row_already_exists
                                                               "Row with the requested ID 'dup1' already
                                                                exists. Try again with a different ID"
tables_db_create_row {books badcol, nonexistent_col}       → ERROR 400 row_invalid_structure
                                                               "Unknown attribute: nonexistent_col"
tables_db_create_row {books badtype, rating "not-a-number"}→ ERROR 400 row_invalid_structure
                                                               "Attribute rating has invalid type.
                                                                Value must be a valid float"
tables_db_create_row {authors dup3, email ada@…}   CONTROL → ERROR 409 same misleading message, new ID
tables_db_create_row {authors dup4, unique email}  CONTROL → OK  ⇒ ID was never the problem
tables_db_delete_row {dup4}                                → cleanup

Appwrite — Pass 2 (fresh namespace, informed)

tables_db_create {bench2}                                  → OK
tables_db_create_table {authors, columns+indexes}          → OK  (1 call, first try)
tables_db_create_table {books, no relationship}            → OK  (1 call, first try)
tables_db_create_rows {authors, 5 rows}                    → OK  total 5, ONE call
tables_db_create_rows {books, 50 rows}                     → OK  total 50, ONE call   ★ bulk works
tables_db_create_relationship_column {books→authors}       → OK  added after bulk, no error
tables_db_list_rows {T7 query}                             → total 20, same 10 IDs, same order,
                                                              still no column values (reproducible)
search_tools "create API key"                              → project_create_ephemeral_key
project_create_ephemeral_key {read scopes, 900s}           → OK (for attribution test below)

Supabase — Pass 1

list_projects {}                                           → 4 projects, all INACTIVE
get_organization {}                          [my omission] → ZodError: expected string, path ["id"]
get_organization {cogxybpczzfqoglfphrm}                    → plan free
get_cost {project}                                         → $0 monthly
confirm_cost {0, monthly}                                  → confirmation id
create_project {mcp-bench-supabase, eu-central-1}          → ACTIVE_HEALTHY immediately
apply_migration t2_create_authors                          → {success:true}
apply_migration t3_create_books                            → {success:true}
apply_migration t4_books_author_fk                         → {success:true}
execute_sql × 5  insert authors … returning id             → ids 1..5
execute_sql  insert 50 books (single statement)            → OK, ONE call
execute_sql  T7 filtered+sorted+paginated                  → 10 rows WITH values, correct order
execute_sql  update books id=25 rating=4.95 returning      → [{id:25,rating:4.95}]
execute_sql  delete books id=50 returning                  → [{id:50}]
execute_sql  count verify                                  → remaining 49, should_be_zero 0
execute_sql  join books×authors limit 5                    → titles + author names, ONE call
apply_migration t11_books_rls (enable RLS + policy)        → {success:true}
get_project_url / get_publishable_keys                     → url + anon key
get_advisors {security}                                    → ERROR rls_disabled_in_public on
                                                              public.authors + remediation URL
search_docs "storage upload"                               → docs exist; no MCP tools for Storage
execute_sql  duplicate email                               → ERROR 23505 duplicate key value violates
                                                              unique constraint "authors_email_key"
                                                              DETAIL: Key (email)=(ada@example.com)
execute_sql  unknown column                                → ERROR 42703 column "nonexistent_col" of
                                                              relation "books" does not exist + caret
execute_sql  bad type                                      → ERROR 22P02 invalid input syntax for type
                                                              double precision: "not-a-number" + caret

Supabase — Pass 2 (fresh schema, reverse order — Supabase first)

apply_migration p2_t2_authors (create schema + authors)    → {success:true}
apply_migration p2_t3_books                                → {success:true}
execute_sql  5 authors + 50 books + FK + count  ONE CALL   → {books:50, authors:5}   ★ full seed, 1 RT
execute_sql  T7 query                                      → identical 10 rows, identical order
execute_sql  duplicate email                               → ERROR 23505, identical to Pass 1

Out of band (curl, not through MCP)

GET supabase /rest/v1/books   anon key    → 200  []                      blocked, no leak
GET supabase /rest/v1/authors anon key    → 200  [3 rows with emails]   CONTROL: exposed
GET appwrite /v1/…/books/rows guest       → 401  user_unauthorized      blocked
GET appwrite /v1/…/authors/rows guest     → 401  user_unauthorized      CONTROL: deny by default

ATTRIBUTION TEST — appwrite REST list-rows with scoped ephemeral key:
GET /v1/tablesdb/bench/tables/books/rows?queries[0]=…&queries[1]=limit:2
  → {"total":33,"rows":[{"title":"Book 01","rating":0.7,"published":true,
      "created":"2025-01-02T00:00:00.000+00:00","$id":"b01",…,"author":"a1",…}]}
  ⇒ REST returns full column data. The MCP server strips it. Defect is in the MCP layer.

LATENCY — 12 samples each, curl time_total, single client:
  appwrite  fra          median 309 ms   p25 291   p75 379   min 278   max 441
  supabase  eu-central-1 median 383 ms   p25 350   p75 721   min 338   max 986