Skip to content

[Bug]: list_schemes tool appears to mention unavailable simulators as simulatorName examples #123

New issue
New issue
Open
Open
[Bug]: list_schemes tool appears to mention unavailable simulators as simulatorName examples#123
Labels
bugSomething isn't workingstale
@ariccio

Description

@ariccio
ariccio
opened on Oct 10, 2025

Bug Description

This is a relatively small issue that seems to degrade claude's performance. There are currently the following simulators available on my system, which I just directed claude to discover for completeness in the bug report here:

⏺ Bash(xcrun simctl list devices available --json | python3 -c "import sys, json; devices = json.load(sys.stdin)['devices']; [(print(f'{runtime}: {len(sims)} simulators'), [print(f'  - {s[\"name\"]} ({s[\"udid\"]})') for s in sims[:3]])
       for runtime, sims in devices.items() if sims]")
  ⎿  com.apple.CoreSimulator.SimRuntime.watchOS-26-0: 5 simulators          
       - Apple Watch Series 11 (46mm) (DA464FE5-E651-4215-A83C-61EE68B4346B)
       - Apple Watch Series 11 (42mm) (536D3DC1-2A38-4262-8011-E2EA8A090FF7)
       - Apple Watch Ultra 3 (49mm) (F12C2ACE-23FE-48CD-AE9F-71157E84DA21)
     com.apple.CoreSimulator.SimRuntime.iOS-26-0: 3 simulators
       - iPhone 17 Pro (0C23D6BE-36CA-46CC-85D5-1F79AEECD9EA)
       - iPad Pro 11-inch (M4) (C099563E-55DD-4D42-BE9C-4ABE1F46A5F9)
       - iPad Pro 13-inch (M4) (151DE739-5D0C-407B-BF75-9518A8F4C3D0)

⏺ Bash(cd "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype" && xcodebuild -project "DeeDee Health.xcodeproj" -scheme "DeeDee Health" -showdestinations 2>&1 | grep -A 1 "Available destinations")
  ⎿  	Available destinations for the "DeeDee Health" scheme:                                      
     		{ platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }

⏺ Bash(if [ -f /Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/scripts/.build/debug/simulator-selector ]; then /Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/scripts/.build/debug/simulator-selector --list; else echo
      "simulator-selector tool not built"; fi)
  ⎿  Available simulators:                                                             
       - Apple Watch Series 11 (46mm) (DA464FE5-E651-4215-A83C-61EE68B4346B) [Shutdown]
       - Apple Watch Series 11 (42mm) (536D3DC1-2A38-4262-8011-E2EA8A090FF7) [Shutdown]
       - Apple Watch Ultra 3 (49mm) (F12C2ACE-23FE-48CD-AE9F-71157E84DA21) [Shutdown]
       - Apple Watch SE 3 (44mm) (22952200-5D64-4745-BEFB-4405AD6AE907) [Shutdown]
       - Apple Watch SE 3 (40mm) (5E402DB0-958E-4958-B20B-CED39BE0BD1E) [Shutdown]
       - iPhone 17 Pro (0C23D6BE-36CA-46CC-85D5-1F79AEECD9EA) [Shutdown]
       - iPad Pro 11-inch (M4) (C099563E-55DD-4D42-BE9C-4ABE1F46A5F9) [Shutdown]
       - iPad Pro 13-inch (M4) (151DE739-5D0C-407B-BF75-9518A8F4C3D0) [Shutdown]

However, list_schemes seems to suggest to claude that it should use simulatorName: "iPhone 16":

     XcodeBuildMCP - list_schemes (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj")
     ✅ Available schemes:                                                                                                                                                                                                                   
     DeeDee Health
     DeeDee Health release?
     Next Steps:
     1. Build the app: build_macos({ projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health" })
        or for iOS: build_sim({ projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health", simulatorName: "iPhone 16" })
     2. Show build settings: show_build_settings({ projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health" })
     XcodeBuildMCP - test_sim (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health", simulatorName: "iPhone 16")
     Error: ❌ [stderr] xcodebuild: error: Unable to find a device matching the provided destination specifier:
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, OS:latest, name:iPhone 16 }
     ---
     ❌ [stderr] 	The requested device could not be found because no available devices matched the request.
     ---
     ❌ [stderr] 	Available destinations for the "DeeDee Health" scheme:
     ---
     ❌ [stderr] 		{ platform:iOS, arch:arm64, id:00008110-001169D0112B801E, name:Alexander’s iPhone }
     ---
     ❌ [stderr] 		{ platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, arch:arm64, id:C099563E-55DD-4D42-BE9C-4ABE1F46A5F9, OS:26.0, name:iPad Pro 11-inch (M4) }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, arch:arm64, id:151DE739-5D0C-407B-BF75-9518A8F4C3D0, OS:26.0, name:iPad Pro 13-inch (M4) }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, arch:arm64, id:0C23D6BE-36CA-46CC-85D5-1F79AEECD9EA, OS:26.0, name:iPhone 17 Pro }
     ---
     ❌ Test Run test failed for scheme DeeDee Health.
     ---

I suspect this contributed to reasoning degeneration in anthropics/claude-code#9318

Debug Output

XcodeBuildMCP Doctor

Generated: 2025-10-10T19:29:28.886Z
Server Version: 1.14.1

System Information

  • platform: darwin
  • release: 25.0.0
  • arch: arm64
  • cpus: 8 x Apple M2
  • memory: 24 GB
  • hostname: Alexanders-MacBook-Pro.local
  • username: alexanderriccio
  • homedir: /Users/alexanderriccio
  • tmpdir: /var/folders/_4/zrj460q97l5cctkts4s8srw00000gn/T

Node.js Information

  • version: v20.19.4
  • execPath: /Users/alexanderriccio/.nvm/versions/node/v20.19.4/bin/node
  • pid: 48588
  • ppid: 48296
  • platform: darwin
  • arch: arm64
  • cwd: /Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype
  • argv: /Users/alexanderriccio/.nvm/versions/node/v20.19.4/bin/node /Users/alexanderriccio/.npm/_npx/99336612077b7094/node_modules/.bin/xcodebuildmcp

Xcode Information

  • version: Xcode 26.0.1 - Build version 17A400
  • path: /Applications/Xcode.app/Contents/Developer
  • selectedXcode: /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild
  • xcrunVersion: xcrun version 72.

Dependencies

  • axe: ✅ Bundled
  • xcodemake: ❌ Not found
  • mise: ❌ Not found

Environment Variables

  • INCREMENTAL_BUILDS_ENABLED: (not set)
  • DEVELOPER_DIR: (not set)
  • HOME: /Users/alexanderriccio
  • USER: alexanderriccio
  • TMPDIR: /var/folders/_4/zrj460q97l5cctkts4s8srw00000gn/T/
  • NODE_ENV: (not set)
  • SENTRY_DISABLED: (not set)
  • XCODEBUILDMCP_ENABLED_WORKFLOWS: simulator,project-discovery,ui-testing,logging,utilities,doctor
  • XCODEBUILDMCP_DYNAMIC_TOOLS: false
  • XCODEBUILDMCP_SILENCE_LOGS: true

PATH

/Users/alexanderriccio/.npm/_npx/99336612077b7094/node_modules/.bin
/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/node_modules/.bin
/Users/alexanderriccio/Documents/GitHub/node_modules/.bin
/Users/alexanderriccio/Documents/node_modules/.bin
/Users/alexanderriccio/node_modules/.bin
/Users/node_modules/.bin
/node_modules/.bin
/Users/alexanderriccio/.nvm/versions/node/v20.19.4/lib/node_modules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin
/opt/homebrew/opt/bison/bin
/Users/alexanderriccio/Library/Android/sdk/platform-tools
/usr/local/opt/postgresql@14/bin
/Users/alexanderriccio/.pyenv/shims
/Users/alexanderriccio/.pyenv/bin
/Users/alexanderriccio/.nvm/versions/node/v20.19.4/bin
/Users/alexanderriccio/.rbenv/shims
/usr/local/bin
/Library/Frameworks/Python.framework/Versions/3.12/bin
/opt/homebrew/bin
/opt/homebrew/sbin
/usr/local/bin
/System/Cryptexes/App/usr/bin
/usr/bin
/bin
/usr/sbin
/sbin
/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin
/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin
/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
/Library/Apple/usr/bin
/Library/TeX/texbin
/Applications/nmap.app/Contents/Resources/bin
/Applications/ncat.app/Contents/Resources/bin
/Applications/nping.app/Contents/Resources/bin
/Applications/zenmap.app/Contents/Resources/bin
/opt/homebrew/opt/bison/bin
/Users/alexanderriccio/Library/Android/sdk/platform-tools
/usr/local/opt/postgresql@14/bin
/Users/alexanderriccio/.cargo/bin
/Applications/Visual Studio Code.app/Contents/Resources/app/bin
/Applications/Visual Studio Code.app/Contents/Resources/app/bin
/Users/alexanderriccio/.local/bin
/Applications/Visual Studio Code.app/Contents/Resources/app/bin
/Users/alexanderriccio/.local/bin
/Applications/Visual Studio Code.app/Contents/Resources/app/bin

Feature Status

UI Automation (axe)

  • Available: ✅ Yes
  • UI Automation Supported: ✅ Yes

Incremental Builds

  • Enabled: ❌ No
  • Available: ❌ No
  • Makefile exists: ❌ No

Mise Integration

  • Running under mise: ❌ No
  • Mise available: ❌ No

Available Tools

  • Total Plugins: 83
  • Plugin Directories: 12
  • device: 14 tools
  • discovery: 1 tools
  • doctor: 1 tools
  • logging: 4 tools
  • macos: 11 tools
  • project-discovery: 5 tools
  • project-scaffolding: 2 tools
  • simulator: 19 tools
  • simulator-management: 8 tools
  • swift-package: 6 tools
  • ui-testing: 11 tools
  • utilities: 1 tools

Runtime Tool Registration

  • Mode: static
  • Enabled Workflows: 12
  • Registered Tools: 61
  • Workflows: device, discovery, doctor, logging, macos, project-discovery, project-scaffolding, simulator, simulator-management, swift-package, ui-testing, utilities

Tool Availability Summary

  • Build Tools: ✅ Available
  • UI Automation Tools: ✅ Available
  • Incremental Build Support: ❌ Not available

Sentry

  • Sentry enabled: ✅ Yes

Troubleshooting Tips

  • If UI automation tools are not available, install axe: brew tap cameroncooke/axe && brew install axe
  • If incremental build support is not available, you can download the tool from https://github.com/cameroncooke/xcodemake. Make sure it's executable and available in your PATH
  • To enable xcodemake, set environment variable: export INCREMENTAL_BUILDS_ENABLED=1
  • For mise integration, follow instructions in the README.md file

### Editor/Client

Claude Code 2.0.13

### MCP Server Version

1.14.1

### LLM

Claude 4.5 Sonnet

### MCP Configuration

```json
{
  "mcpServers": {
    "context7": {
      "type": "http",
      "url": "https://mcp.context7.com/mcp",
      "tools": [
        "get-library-docs",
        "resolve-library-id"
      ]
    },
    "XcodeBuildMCP": {
      "command": "npx",
      "args": [
        "-y",
        "xcodebuildmcp@latest"
      ],
      "env": {
        "XCODEBUILDMCP_ENABLED_WORKFLOWS": "simulator,project-discovery,ui-testing,logging,utilities,doctor",
        "XCODEBUILDMCP_DYNAMIC_TOOLS": "false"
      }
    },
    "apple-docs": {
      "command": "./scripts/run-apple-docs-mcp.sh",
      "args": []
    }
  }
}

Steps to Reproduce

Here's the subagent Task that claude code invoked:

  ⎿  Prompt:
       You are working in the DeeDee-Prototype iOS app repository following the instructions at `.github/copilot-instructions.md`.

       **CONTEXT**: Just implemented CRITICAL fix for alert validation false positives:
       - **Problem**: 100% false positive rate (expected ≤30%)
       - **Root Cause**: Used OR logic (alert if ANY single metric concerning)
       - **Fix Applied**: Multi-metric correlation (alert only if ≥2 metrics concerning)
       - **File Modified**: `DeeDee HealthTests/AlertValidationTests.swift` lines 251-271

       **YOUR TASK**: Run AlertValidationTests to verify false positive rate drops to ≤30%.

       **INSTRUCTIONS**:
       1. Use XcodeBuildMCP `test_sim` to run ONLY the AlertValidationTests:
          - projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Prototype.xcodeproj"
          - scheme: "DeeDee Prototype"
          - simulatorName: "iPhone 16"
          - NOTE: You may need to specify test targets/classes if the tool supports filtering

       2. Look for test output showing:
          - `testNormalPatterns()` result
          - False positive rate metric (should be ≤30%)
          - Validation report with precision/recall/F1-score

       3. Report results CONCISELY:
          - **Success**: "✓ Alert validation fixed. False positive rate: X% (≤30% target met)"
          - **Partial Success**: "⚠ Improved but not target. False positive rate: X% (target: ≤30%)"
          - **Failure**: "✗ Still failing. False positive rate: X%. Details: [error summary]"

       4. DO NOT read full test logs into context - extract only the validation metrics

       **CRITICAL**: Keep output under 200 tokens. Follow `copilot_notes/build-operations-canonical.md` token management.

       Execute the test verification now and report validation metrics concisely.

     XcodeBuildMCP - test_sim (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Prototype.xcodeproj", scheme: "DeeDee Prototype", simulatorName: "iPhone 16")
     Error: ❌ [stderr] xcodebuild: error: '/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Prototype.xcodeproj' does not exist.
     ---
     ❌ Test Run test failed for scheme DeeDee Prototype.
     ---

     Test Results Summary:
     Test Summary: Test - T
     Overall Result: unknown

     Test Counts:
       Total: 0
       Passed: 0
       Failed: 0
       Skipped: 0
       Expected Failures: 0

     Environment: T
     Bash(find /Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype -name "*.xcodeproj" -maxdepth 2)
     Waiting…
     /Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj
     XcodeBuildMCP - test_sim (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Prototype", simulatorName: "iPhone 16")
     Error: ❌ [stderr] xcodebuild: error: The project named "DeeDee Health" does not contain a scheme named "DeeDee Prototype". The "-list" option can be used to find the names of the schemes in the project.
     ---
     ❌ Test Run test failed for scheme DeeDee Prototype.
     ---

     Test Results Summary:
     Test Summary: Test - DeeDee Health.xcodeproj
     Overall Result: unknown

     Test Counts:
       Total: 0
       Passed: 0
       Failed: 0
       Skipped: 0
       Expected Failures: 0

     Environment: DeeDee Health.xcodeproj
     XcodeBuildMCP - list_schemes (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj")
     ✅ Available schemes:                                                                                                                                                                                                                   
     DeeDee Health
     DeeDee Health release?
     Next Steps:
     1. Build the app: build_macos({ projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health" })
        or for iOS: build_sim({ projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health", simulatorName: "iPhone 16" })
     2. Show build settings: show_build_settings({ projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health" })
     XcodeBuildMCP - test_sim (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health", simulatorName: "iPhone 16")
     Error: ❌ [stderr] xcodebuild: error: Unable to find a device matching the provided destination specifier:
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, OS:latest, name:iPhone 16 }
     ---
     ❌ [stderr] 	The requested device could not be found because no available devices matched the request.
     ---
     ❌ [stderr] 	Available destinations for the "DeeDee Health" scheme:
     ---
     ❌ [stderr] 		{ platform:iOS, arch:arm64, id:00008110-001169D0112B801E, name:Alexander’s iPhone }
     ---
     ❌ [stderr] 		{ platform:iOS, id:dvtdevice-DVTiPhonePlaceholder-iphoneos:placeholder, name:Any iOS Device }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, id:dvtdevice-DVTiOSDeviceSimulatorPlaceholder-iphonesimulator:placeholder, name:Any iOS Simulator Device }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, arch:arm64, id:C099563E-55DD-4D42-BE9C-4ABE1F46A5F9, OS:26.0, name:iPad Pro 11-inch (M4) }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, arch:arm64, id:151DE739-5D0C-407B-BF75-9518A8F4C3D0, OS:26.0, name:iPad Pro 13-inch (M4) }
     ---
     ❌ [stderr] 		{ platform:iOS Simulator, arch:arm64, id:0C23D6BE-36CA-46CC-85D5-1F79AEECD9EA, OS:26.0, name:iPhone 17 Pro }
     ---
     ❌ Test Run test failed for scheme DeeDee Health.
     ---

     Test Results Summary:
     Test Summary: Test - DeeDee Health
     Overall Result: unknown

     Test Counts:
       Total: 0
       Passed: 0
       Failed: 0
       Skipped: 0
       Expected Failures: 0

     Environment: DeeDee Health
     XcodeBuildMCP - test_sim (MCP)(projectPath: "/Users/alexanderriccio/Documents/GitHub/DeeDee-Prototype/DeeDee Health.xcodeproj", scheme: "DeeDee Health", simulatorId: "0C23D6BE-36CA-46CC-85D5-1F79AEECD9EA")

Expected Behavior

Pick/suggest any of:
- iPhone 17 Pro (0C23D6BE-36CA-46CC-85D5-1F79AEECD9EA)
- iPad Pro 11-inch (M4) (C099563E-55DD-4D42-BE9C-4ABE1F46A5F9)
- iPad Pro 13-inch (M4) (151DE739-5D0C-407B-BF75-9518A8F4C3D0)

...for list_schemes

Actual Behavior

Suggested iPhone 16 in list_schemes.

Error Messages

Metadata
Metadata
Assignees
No one assigned
    Labels
    bugSomething isn't workingstale
    Projects
    No projects
    Milestone
    No milestone
    Relationships
    None yet
    Development
    No branches or pull requests
    Issue actions