Scorecard MCP 2.0: 1,000 Lines -> 70

Following our launch of the first remote MCP server for evaluation, we're excited to introduce an even better 2.0 version. The new MCP specification dropped June 18th with game-changing updates: fixed authentication, elicitation for richer AI interactions, and structured tool outputs.

We partnered with Clerk to ship one of the first production implementations. The result? 1000 lines of auth code reduced to just 70.

Why the New MCP Spec Is a Big Deal

The MCP community has been vocal about what changed. "Auth being fixed is HUGE" was a common refrain - finally, there's a standardized way to handle OAuth. The spec also introduces elicitation (servers can ask users for clarification) and structured outputs for better AI reasoning. For context on the auth challenges that were solved, see this analysis and the GitHub discussion.

The Dramatic Simplification

Our original implementation required custom OAuth flows, manual token management, and complex state handling. The new MCP spec with standardized authentication reduced our implementation significantly. 

With Clerk's MCP integration and Vercel's adapter, the complete MCP route handler is now 70 lines (down from 1000+) and our current auth implementation is now:

// Authentication wrapper - Clerk handles OAuth complexity
const authHandler = withMcpAuth(
  handler,
  async (_, token) => {
    const clerkAuth = await auth({ acceptsToken: "oauth_token" });
    return verifyClerkToken(clerkAuth, token);
  },
  { required: true },
);

// Tool handlers get authenticated client automatically
async (params, context) => {
  const accessToken = params.authInfo.token;
  const client = new Scorecard({
    baseURL: process.env.NEXT_PUBLIC_API_URL + "/api/v2",
    fetch: async (url, init) => {
      const headers = new Headers(init?.headers);
      headers.set("Authorization", `Bearer ${accessToken}`);
      return fetch(url, { ...init, headers });
    },
  });
  return endpoint.handler(client, params);
}

How We Built It

The implementation builds on three key technologies:‍

Clerk handled the authentication complexity. Their new MCP tools provide enterprise-grade OAuth with PKCE, automatic token refresh, and dynamic client registration. What previously required hundreds of lines of custom code now works out of the box.

Vercel's MCP adapter unified the transport layer. It handles both legacy SSE connections (for older clients) and the new HTTP transport, with built-in Redis support for stateful connections when needed.

Stainless continues to power our API-to-MCP transformation, automatically generating type-safe bindings from our OpenAPI spec. One line still exposes our entire evaluation API through MCP.

Getting Started

For Scorecard users, connecting is now simpler than ever. Replace the placeholder MCP_SERVER_URL in your MCP.json file with the actual URL value provided on GitHub e.g.:

{
  "mcpServers": {
    "scorecard": {
      "url": "MCP_SERVER_URL"
    }
  }
}

Note: Claude web/desktop and Cline are not currently supported. We’re working with Anthropic and the Clerk team to add support soon.

What's Next for Scorecard MCP

We're incredibly excited by how the new spec has lowered the friction of building and deploying MCP servers. Scorecard's evaluation tools are now accessible from any MCP client with minimal configuration and rock-solid authentication.

For developers interested in the implementation details, check out our open source code, Clerk's MCP demo, and Vercel's adapter. The future of AI development is getting simpler, and we're here for it.