Server Quality Checklist

Tool Scores

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Create a draft order' implies a write/mutation operation, but it doesn't disclose any behavioral traits: no information about permissions required, whether the creation is reversible, rate limits, side effects, or what happens upon success/failure. For a mutation tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just three words, with zero wasted language. It's front-loaded with the core action and resource. While this conciseness contributes to under-specification in other dimensions, as a standalone assessment of brevity and structure, it's maximally efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters including nested objects, no output schema, and no annotations), the description is incomplete. It doesn't explain what a draft order is, how it differs from a completed order, what the tool returns, or any behavioral context. For a mutation tool that creates a complex resource, the description fails to provide adequate context for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with clear descriptions for each parameter (e.g., 'Line items to add to the order', 'Customer email'). The description adds no additional meaning beyond what the schema already provides. According to the rules, when schema coverage is high (>80%), the baseline score is 3 even with no parameter info in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a draft order' is a tautology that essentially restates the tool name 'create-draft-order'. While it indicates the action (create) and resource (draft order), it doesn't provide any additional specificity about what a draft order entails or how it differs from other order-related tools like 'complete-draft-order' or 'get-order'. The purpose is stated but lacks meaningful differentiation from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether customer or product data must exist first), when to choose this over 'complete-draft-order', or any constraints on usage. With siblings like 'complete-draft-order' and 'get-order', the absence of comparative guidance leaves the agent guessing about appropriate contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the action without behavioral details. It doesn't disclose if this is a mutation (likely), requires specific permissions, has side effects (e.g., order becomes active), or any rate limits. This is inadequate for a tool that presumably changes state.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero waste—'Complete a draft order' is front-loaded and appropriately sized for the tool's purpose. No unnecessary words or structure detract from clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a likely mutation tool, the description is incomplete. It doesn't explain what 'complete' does behaviorally, what happens after completion, or any error conditions. For a tool with two required parameters and potential state changes, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (draftOrderId, variantId). The description adds no meaning beyond this, such as explaining why variantId is required or how IDs are formatted. Baseline 3 is appropriate as the schema handles parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Complete a draft order' states a clear verb ('complete') and resource ('draft order'), but it's vague about what 'complete' entails—does it finalize, process, or submit the order? It distinguishes from siblings like 'create-draft-order' by implying a different stage, but lacks specificity on the action's outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing draft order), exclusions, or related tools like 'get-order' for post-completion checks, leaving the agent to infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get shop details' implies a read-only operation, but it doesn't specify aspects like authentication requirements, rate limits, error conditions, or what the return data includes. This leaves significant gaps for an agent to understand how to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words, making it front-loaded and efficient. However, it may be overly brief, potentially under-specifying the tool's purpose compared to siblings like 'get-shop-details', which could confuse an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a tool that likely retrieves shop information, the description is incomplete. With no annotations, no output schema, and a vague purpose, it fails to provide enough context for an agent to use it correctly, especially in distinguishing it from similar sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description doesn't add parameter details, but since there are no parameters, this is acceptable. The baseline for 0 parameters is 4, as the description doesn't need to compensate for missing param info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get shop details' clearly states the action (get) and resource (shop details), making the purpose understandable. However, it's somewhat vague about what specific details are retrieved, and it doesn't distinguish this tool from 'get-shop-details' among the sibling tools, which appears to serve a similar function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. Given the sibling tool 'get-shop-details', there's a clear need for differentiation, but the description offers no context, prerequisites, or exclusions to help an agent choose appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states 'Create' which implies a write/mutation operation, but doesn't mention permissions required, whether the discount is immediately active, if there are rate limits, or what happens on success/failure. For a creation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a tool with well-documented parameters in the schema. Every word earns its place by clearly stating the tool's core function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a creation tool with no annotations and no output schema, the description is insufficient. It doesn't explain what happens after creation (e.g., returns a discount ID, activates immediately), doesn't mention error conditions or validation rules, and provides no context about the discount system's limitations. The 100% schema coverage helps with parameters but doesn't compensate for missing behavioral and outcome information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all 7 parameters with clear descriptions. The description adds no additional parameter information beyond what's in the schema. According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Create') and resource ('a basic discount code'), making the purpose immediately understandable. It distinguishes this from sibling tools like 'complete-draft-order' or 'get-products' by focusing on discount creation rather than order or product operations. However, it doesn't specify what 'basic' means compared to more advanced discount options that might exist.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There are no explicit instructions about prerequisites, constraints, or comparisons with other discount-related tools (if any existed in the sibling list). The agent must infer usage from the tool name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get all collections' implies a read operation, but it doesn't specify permissions needed, rate limits, pagination behavior, or what 'all' entails (e.g., across the entire system). The description lacks details on return format or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just three words, front-loading the core purpose without any wasted text. Every word earns its place, making it efficient for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool with parameters. It doesn't explain what 'collections' are in this context, how results are structured, or behavioral aspects like ordering or defaults. For a list operation with filtering parameters, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the 'limit' and 'name' parameters. The description adds no parameter-specific information beyond what's in the schema, but since the schema is comprehensive, a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get all collections' clearly states the verb ('Get') and resource ('collections'), making the tool's purpose immediately understandable. It doesn't distinguish from sibling tools like 'get-products-by-collection', but it's specific enough to avoid vagueness or tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get-products-by-collection' for filtering products by collection, or clarify if this is for listing all collections versus filtered subsets. No exclusions or prerequisites are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions 'pagination support' which hints at behavior for large result sets, but fails to disclose critical details: whether this is a read-only operation, what permissions are required, rate limits, error conditions, or the format of returned data. For a tool with no annotations, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and includes a key behavioral hint (pagination). Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a data retrieval tool. It doesn't explain what data is returned (e.g., customer fields), how pagination works in practice, or error handling. For a tool with 2 parameters and complex behavior implied by pagination, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters ('limit' and 'next'). The description adds no additional meaning about parameters beyond implying pagination through 'next', which is already clear from the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('shopify customers'), making the purpose understandable. It distinguishes from siblings like 'tag-customer' or 'get-order' by specifying customers as the target. However, it doesn't explicitly differentiate from other customer-related tools (none listed), so it's not a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'pagination support' which implies usage for large datasets, but provides no explicit guidance on when to use this tool versus alternatives like 'get-order' or 'get-products'. No prerequisites, exclusions, or comparisons to sibling tools are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves an order but fails to describe key behaviors: whether it's a read-only operation, what happens if the order ID is invalid (e.g., error handling), or any rate limits or authentication requirements. This leaves significant gaps in understanding how the tool behaves in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded, consisting of a single, clear sentence that directly states the tool's purpose. There is no wasted verbiage, and it efficiently communicates the core functionality without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a retrieval tool. It doesn't explain what is returned (e.g., order details, error formats) or address behavioral aspects like error conditions. While the purpose is clear, the overall context for effective tool use is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'orderId' fully documented in the schema as 'ID of the order to retrieve'. The description adds no additional semantic context beyond implying the parameter is required for retrieval, so it meets the baseline score without compensating for or enhancing the schema information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('a single order by ID'), making the purpose immediately understandable. It distinguishes this from sibling tools like 'get-orders' (plural) by specifying retrieval of a single order, though it doesn't explicitly contrast with other sibling tools beyond this implicit differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-orders' or other retrieval tools. It lacks context about prerequisites, such as needing an existing order ID, and offers no explicit when-not-to-use or alternative recommendations, leaving usage decisions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'advanced filtering and sorting' but doesn't explain what that entails operationally. It doesn't disclose whether this is a read-only operation, what permissions are required, whether there are rate limits, pagination behavior (implied by 'first' and 'after' parameters but not stated), or what the output format looks like. For a tool with 5 parameters and no output schema, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose without unnecessary words. It's appropriately sized for a tool with this complexity level and gets straight to the point. Every word earns its place, with no redundant information or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't explain what 'advanced filtering' means operationally, doesn't describe the return format or structure, and provides no guidance on error conditions or limitations. For a data retrieval tool with multiple filtering/sorting options, users need more context about what to expect when invoking it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds 'advanced filtering and sorting' which aligns with the 'query' and 'sortKey'/'reverse' parameters, providing some high-level context. However, it doesn't add specific meaning beyond what the schema provides, such as explaining query syntax examples or typical sortKey usage patterns.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('shopify orders'), making the purpose evident. It adds 'with advanced filtering and sorting' which provides useful context about capabilities. However, it doesn't explicitly differentiate from sibling tools like 'get-order' (singular) or 'get-customers', leaving some ambiguity about when to choose this specific tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to use 'get-orders' versus 'get-order' (singular), nor does it indicate prerequisites, context requirements, or typical use cases. The phrase 'advanced filtering and sorting' hints at capabilities but doesn't specify when those features are needed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states what the tool does but lacks details on permissions, rate limits, pagination, error handling, or response format. For a read operation with no annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just one sentence that efficiently conveys the core functionality. It's front-loaded and wastes no words, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the return values look like, how results are structured, or any behavioral constraints. For a tool with two parameters and no structured output documentation, more context is needed for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value by implying the 'searchTitle' parameter's optional nature and the tool's dual functionality, but doesn't provide additional syntax or format details beyond what the schema specifies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('products'), and distinguishes between two modes: retrieving all products or searching by title. However, it doesn't explicitly differentiate from sibling tools like 'get-products-by-collection' or 'get-products-by-ids', which would require a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-products-by-collection' or 'get-products-by-ids'. It mentions the two modes of operation but doesn't specify scenarios, prerequisites, or exclusions for choosing this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic action. It doesn't mention whether this is a read-only operation, potential rate limits, authentication needs, error conditions, or response format (e.g., pagination, data structure). For a tool with zero annotation coverage, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for the tool's complexity, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a retrieval tool. It lacks details on behavioral traits (e.g., read-only nature, error handling), response format, or usage distinctions from siblings. While concise, it doesn't provide enough context for an agent to fully understand how to invoke and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with clear documentation for both parameters (collectionId and limit). The description adds no additional semantic context beyond what the schema provides, such as example values or constraints not in the schema. This meets the baseline score of 3 when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('products from a specific collection'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get-products' or 'get-products-by-ids', which would require more specific language about collection-based filtering versus other retrieval methods.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-products' or 'get-products-by-ids'. It mentions 'specific collection' but doesn't clarify prerequisites, exclusions, or comparative contexts with sibling tools, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic action. It doesn't cover critical aspects like whether this is a read-only operation, error handling, rate limits, authentication needs, or what happens with invalid IDs, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single, direct sentence that front-loads the core functionality. There is no wasted language, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain return values, error conditions, or behavioral nuances needed for a tool that retrieves data by IDs, failing to compensate for the missing structured information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the single parameter 'productIds'. The description adds no additional meaning beyond what the schema provides, such as format examples or constraints, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('products by their IDs'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get-products' or 'get-products-by-collection', which would require more specificity to earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-products' or 'get-products-by-collection'. It lacks context about prerequisites, limitations, or typical use cases, offering only basic functional information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states what the tool does ('Get product variants'), but doesn't describe whether this is a read-only operation, what permissions are required, how many variants can be retrieved at once, error handling for invalid IDs, or the format of returned data. For a retrieval tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero wasted words. It follows the principle of front-loading the most important information (what the tool does) without unnecessary elaboration. Every word earns its place in this minimal but complete statement of purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations, no output schema, and the description provides only basic purpose without behavioral context, this is incomplete for effective agent use. While the purpose is clear, the agent lacks information about what the tool returns, how it handles errors, limitations, or authentication requirements. For a data retrieval tool, this represents significant contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100% with the single parameter 'variantIds' fully documented in the schema. The description adds no additional parameter semantics beyond what's already in the schema ('Array of variant IDs to retrieve'). According to scoring rules, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Get product variants by their IDs' clearly states the verb ('Get'), resource ('product variants'), and key constraint ('by their IDs'). It distinguishes from general product retrieval tools like 'get-products' or 'get-products-by-ids' by focusing specifically on variants. However, it doesn't explicitly differentiate from potential sibling tools that might also retrieve variants through other means.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when this tool is preferred over 'get-products' (which might include variants) or 'get-products-by-ids', nor does it specify prerequisites like needing specific variant IDs. The agent must infer usage from the tool name and description alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. While it implies mutation (subscribe/unsubscribe) and read (find), it doesn't specify permissions required, rate limits, side effects (e.g., whether unsubscribe destroys data), or response format. For a tool with potential write operations and no annotation coverage, this leaves critical gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (three words) and front-loaded, with zero wasted words. Every term ('subscribe', 'find', 'unsubscribe', 'webhooks') directly contributes to understanding the tool's scope without redundancy. This efficiency makes it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple actions including mutations), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like authentication, error handling, or return values, which are crucial for safe invocation. While concise, it fails to provide enough context for an agent to use the tool effectively without trial and error.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning about parameters beyond implying the three actions map to the 'action' enum. It doesn't clarify parameter interactions (e.g., webhookId is only for unsubscribe) or provide examples, so it meets the baseline but doesn't compensate for schema gaps (there are none).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs (subscribe, find, unsubscribe) and resource (webhooks), making it immediately understandable. However, it doesn't differentiate this multi-action tool from its siblings, which are mostly single-action retrieval tools (e.g., get-products, get-orders) or specific creation tools (e.g., create-draft-order).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., authentication needs), when to choose subscribe vs. find vs. unsubscribe, or how it relates to sibling tools like get-orders (which might overlap with webhook topics). Without this context, an agent must infer usage from parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Add tags' implies a mutation operation, it lacks details on permissions required, whether tags are appended or replaced, error handling (e.g., invalid customerId), rate limits, or response format. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized for a simple tool and front-loaded with the core action, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what happens on success (e.g., returns updated customer object) or failure, nor does it cover behavioral aspects like idempotency or side effects. For a 2-parameter tool with 100% schema coverage but critical gaps in mutation context, a low score is warranted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with both parameters (customerId and tags) clearly documented in the schema. The description adds no additional meaning beyond what the schema provides (e.g., no examples, format details, or constraints). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Add tags') and target resource ('to a customer'), providing a specific verb+resource combination. However, it doesn't differentiate from potential sibling tools like 'get-customers' or other customer management tools that might exist in the broader ecosystem, though none are directly related to tagging in the provided sibling list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., customer must exist), exclusions (e.g., cannot remove tags), or related tools (e.g., if there's a 'remove-tags' or 'update-customer' tool). Usage is implied only by the action itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves data ('Get'), implying a read-only operation, but doesn't address other behavioral aspects such as authentication requirements, rate limits, error conditions, or response format. The description is minimal and lacks critical operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is front-loaded with the core purpose and includes a specific detail ('shipping countries') that adds value. There is no waste or redundancy, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that likely returns complex data. It mentions 'extended shop details' and 'shipping countries' but doesn't explain what other details are included, the response structure, or any limitations. For a read operation with no structured output documentation, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, meaning no parameters are documented in the schema. The description doesn't add parameter details, which is appropriate since there are none to describe. It implies the tool operates without inputs, which aligns with the schema. Baseline is 4 for 0 parameters, as no additional semantic information is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('extended shop details'), including the scope of information returned ('including shipping countries'). It distinguishes this from the sibling 'get-shop' tool by specifying 'extended' details, though it doesn't explicitly contrast them. The purpose is not vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get-shop' or other sibling tools. It lacks context about prerequisites, timing, or exclusions. While the 'extended' detail implies more comprehensive information, it doesn't specify when this is necessary over basic shop details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.