Bring Your Own LLM

Neural Inverse is a BYOLLM (Bring Your Own LLM) IDE. Your API keys stay on your machine. No proxy, no cloud dependency, no telemetry.

Supported Providers (20)

Provider	Type	Connection	OpenAI-Compatible
Anthropic (Claude)	Cloud	API key	No (native SDK)
OpenAI	Cloud	API key	Yes
Google Gemini	Cloud	API key	No (native SDK)
xAI (Grok)	Cloud	API key	Yes
DeepSeek	Cloud	API key	Yes
Mistral	Cloud	API key	Yes
Groq	Cloud	API key	Yes
OpenRouter	Cloud	API key	Yes
GitHub Models	Cloud	GitHub PAT (`models:read`)	Yes
Fireworks AI	Cloud	API key	Yes
Cerebras	Cloud	API key	Yes
AWS Bedrock	Cloud	Credentials + region	Yes (via proxy)
Google Vertex AI	Cloud	Project + region	Yes
Microsoft Azure OpenAI	Cloud	Resource + key	Yes (Azure SDK)
Ollama	Local	localhost (auto-detected)	Yes
vLLM	Local/Cloud	Endpoint URL	Yes
LM Studio	Local	localhost (auto-detected)	Yes
LiteLLM	Gateway	Endpoint URL	Yes
OpenAI-Compatible	Any	Endpoint + API key	Yes

Per-Feature Model Selection

You can assign different models to different features:

Feature	What it does	Recommended
Chat	Sidebar conversation	Powerful model (Claude, GPT-4)
Ctrl+K	Inline quick edit	Powerful model
Autocomplete	Tab completions as you type	Fast model (local preferred)
Apply	Code modification execution	Fast model
SCM	Git commit message generation	Any model
Power Mode	Autonomous agent	Powerful with tool support

Local Provider Auto-Detection

When you have Ollama, vLLM, or LM Studio running locally, Neural Inverse:

Detects the running server on startup (polls well-known localhost ports)
Fetches the available models list
Offers to auto-configure the provider (with undo + "don't ask again" options)
Refreshes every 30 seconds to track model additions/removals

No manual configuration needed for local providers.

Architecture

Browser (Chat UI / Power Mode)
    │
    ▼ IPC channel
Main Process (electron-main)
    │
    ▼ sendLLMMessage.impl.ts
Provider SDK (OpenAI / Anthropic / Gemini native)
    │
    ▼ HTTPS
Provider API (streaming SSE response)
    │
    ▼ events back to browser
Chat UI renders tokens

Key Files

File	Purpose
`src/vs/workbench/contrib/void/common/modelCapabilities.ts`	Provider settings, model catalogs, capabilities
`src/vs/workbench/contrib/void/common/voidSettingsTypes.ts`	Type definitions, display names, UI info
`src/vs/workbench/contrib/void/electron-main/llmMessage/sendLLMMessage.impl.ts`	SDK instantiation, API dispatch

Adding a New Provider (Step-by-Step)

1. Define provider settings

In modelCapabilities.ts, add to defaultProviderSettings:

export const defaultProviderSettings = {
  // ... existing providers
  myProvider: {
    apiKey: '',
    // add endpoint if not fixed, region if applicable
  },
} as const

The ProviderName type derives automatically from Object.keys(defaultProviderSettings).

2. Add default models

In defaultModelsOfProvider:

export const defaultModelsOfProvider = {
  // ... existing
  myProvider: [
    'model-id-1',
    'model-id-2',
  ],
} as const satisfies Record<ProviderName, string[]>

3. Define model capabilities

Create a model options object with VoidStaticModelInfo entries:

const myProviderModelOptions = {
  'model-id-1': {
    contextWindow: 128_000,
    reservedOutputTokenSpace: 8_192,
    cost: { input: 1.00, output: 3.00 },
    downloadable: false,
    supportsFIM: false,
    specialToolFormat: 'openai-style',  // or 'anthropic-style' or 'gemini-style'
    supportsSystemMessage: 'system-role', // or 'developer-role' or 'separated' or false
    reasoningCapabilities: false,  // or reasoning config object
  },
} as const satisfies { [s: string]: VoidStaticModelInfo }

4. Create provider settings object

const myProviderSettings: VoidStaticProviderInfo = {
  modelOptions: myProviderModelOptions,
  modelOptionsFallback: (modelName) => {
    // try extensiveModelOptionsFallback for known open-source models
    const res = extensiveModelOptionsFallback(modelName)
    if (res?.specialToolFormat === 'anthropic-style' || res?.specialToolFormat === 'gemini-style') {
      res.specialToolFormat = 'openai-style' // coerce if provider only speaks openai
    }
    return res
  },
  providerReasoningIOSettings: {
    input: { includeInPayload: openAICompatIncludeInPayloadReasoning },
    // output: { needsManualParse: true } if reasoning comes in think tags
  },
}

Add it to modelSettingsOfProvider.

5. Add display info

In voidSettingsTypes.ts:

displayInfoOfProviderName() — add an else if returning { title: 'My Provider' }
subTextMdOfProviderName() — add a line with a link to get the API key
displayInfoOfSettingName() — add API key placeholder (e.g., 'mk-...')
Add an initial state entry in the defaultSettingsState.settingsOfProvider object

6. Add SDK configuration

In sendLLMMessage.impl.ts, add to newOpenAICompatibleSDK:

else if (providerName === 'myProvider') {
  const thisConfig = settingsOfProvider[providerName]
  return new OpenAI({
    baseURL: 'https://api.myprovider.com/v1',
    apiKey: thisConfig.apiKey,
    ...commonPayloadOpts,
  })
}

7. Add dispatch entry

In sendLLMMessageToProviderImplementation:

myProvider: {
  sendChat: (params) => _sendOpenAICompatibleChat(params),
  sendFIM: null,  // or _sendOpenAICompatibleFIM if FIM supported
  list: null,     // or _openaiCompatibleList if model listing supported
},

8. Verify

npx tsc --noEmit -p src/tsconfig.json

The UI auto-renders the new provider — no React changes needed.

Reasoning Models

Providers handle reasoning differently. Neural Inverse normalizes this:

Pattern	Providers	Config
Effort slider (`reasoning_effort`)	OpenAI, xAI, Groq	`reasoningSlider: { type: 'effort_slider', values: [...] }`
Budget slider (`budget_tokens`)	Anthropic	`reasoningSlider: { type: 'budget_slider', min, max }`
Think tags (`<think>...</think>`)	DeepSeek, Qwen, open-source	`openSourceThinkTags: ['<think>', '</think>']`
Provider-specific payload	Gemini, OpenRouter	Custom `includeInPayload` function

For providers with manual think-tag parsing, set output: { needsManualParse: true } in providerReasoningIOSettings.

Configuration

All provider settings are stored locally in VS Code's settings storage:

API keys are stored in the system keychain via ISecretStorageService
Endpoints, regions, and model selections are in user profile storage
Nothing is sent to any Neural Inverse server

Settings are accessible via Cmd+Shift+P > search "Neural Inverse Settings" or through the gear icon in the sidebar.

Was this page helpful?

On this page