How New API works
A product guide to the New API open-source AI gateway: how requests flow, what you can configure, and where to find the setup instructions.
On this page
How a request flows through New API
When several apps need access to different models, an AI gateway gives you a place to manage connections, access and usage. New API brings these settings into a web console you can host yourself.
Apps and AI agents
Chat apps, development tools and your own services.
New API- Access tokens
- Channel routing
- Usage records
Your model providers
OpenAI, Anthropic, Gemini and compatible model services.
New API checks the access token and selects a channel before sending the request to your configured model provider. The response returns through the gateway to your app.
When do you need an AI gateway?
- Share model access with your team
- Create tokens for people or applications, with model permissions, quotas, expiry dates and IP restrictions. Manage upstream channels in the gateway.
- Connect apps to multiple models
- Point clients at your gateway to use configured models. Check the API documentation for protocol, parameter and model support before integrating with your app.
- Understand requests and costs
- Review requests, token usage and charges in one place. Investigate failed calls and adjust channel priorities, weights and billing settings for your workload.
Capabilities and configuration
Choose the settings your applications need. Each capability links to documentation with setup steps and supported behavior.
Models and protocols
Connect multiple providers through OpenAI, Claude, Gemini and other interfaces, with protocol conversion. Endpoint, parameter and conversion support depends on the model and channel type.
Explore the API referenceRouting and failover
Configure several channels for a model and distribute requests using priorities and weights. Use retry and channel status settings to handle upstream failures.
Configure channelsTokens and access limits
Issue separate application tokens with quotas, expiry dates, model permissions and IP allowlists. Manage access in the console.
Manage access tokensUsage and cost accounting
Inspect model prices and multipliers, and record costs using configured billing rules. Settings cover usage, per-call charges and cached tokens.
Understand pricingRequest records
Review models, token counts, latency and charges for your calls. Filter usage logs to investigate requests and changes in consumption.
Explore usage logsGeneration task plugins
Connect image, video and music generation services through task plugins. Review each plugin's models, configuration and API documentation.
Read the plugin docsFeatures and protocol support change between releases. Check the documentation for your version and test your models, clients and request parameters.
Deployment and integration
Start the gateway, connect a model provider, then issue a token for your application.
- 1
Start a New API instance
Choose Docker, Docker Compose or another documented deployment method. Configure the database and persistent storage.
Choose a deployment method - 2
Add a model channel
Enter an authorized upstream endpoint and credentials, configure available models, then test the connection.
Add and test a channel - 3
Create an application token
Set model permissions and a quota. Enter your gateway address and token in the client, then send a test request.
Create an API token
Frequently asked questions
Does New API include models or API credits?
New API provides gateway software and a management console. Connect your own model providers or compatible endpoints with the required credentials. Hosting and upstream model usage have their own costs.
Where does request data go when I self-host?
The gateway runs on your infrastructure and sends request content to the model provider you configure. Third-party providers still process those requests. For local inference, connect your own compatible model endpoint and check the complete data flow.
How does cached-token billing differ from response caching?
Cached-token billing uses cache usage reported by the provider and your configured multipliers to calculate charges. Gateway response caching stores and reuses complete responses. Check the current documentation and provider behavior when evaluating either capability.
How can I use the open-source version?
New API is distributed under AGPLv3 and the additional terms in the project's LICENSE. Read LICENSE and NOTICE in the repository before deploying, modifying or distributing it. The project documentation includes commercial cooperation and licensing contacts.