Listen to this article. Also available on Spotify. Subscribe to PolyAPI Squawk.
Server functions in PolyAPI are designed to optimize infrastructure usage by defaulting to a sleep state when not in use. This means KNative scales down the container instances (replicas) to zero, conserving compute resources. However, when a server function is invoked from this idle state, it must first spin up—an operation that introduces latency. Faster wakeup times are crucial for improving performance and reducing the need for costlier always-on configurations.
The Cost of Wakeup Latency
When spin-up times are not acceptable, customers have the option to enable “always-on” functions by setting a minimum replica count of one. While this ensures instant availability, it comes at a financial cost in our Platform-as-a-Service (PaaS) offering. Self-hosted, on-premise users do not pay for these credits, but still incur the underlying infrastructure costs. Ultimately, minimizing spin-up latency improves performance and reduces reliance on always-on configurations.
Current Bottleneck: On-Demand SDK Generation
A significant portion of wakeup time is currently spent generating the client SDK dynamically. While this approach offers major advantages—such as the ability to include client functions, substitute function IDs, and inject runtime variables via template parameters without redeployment—it adds overhead during wakeup. Balancing flexibility with performance is a key area of ongoing improvement.
What’s New in Release 20
Release 20 introduces enhancements aimed at reducing SDK generation time:
- –generateContexts Argument: Introduced in a previous release, this allows users to specify a subset of contexts to include, reducing SDK size and generation time.
- –no-types Argument: Newly added in R20, this option skips type generation, which is not required at runtime, further speeding up the wakeup process.
These optimizations enable faster function activation while preserving essential capabilities.
What’s Coming Next
Looking ahead, PolyAPI is working on a major improvement to provide the option to fix the SDK at deploy time. Many server functions do not require dynamic SDK generation, and in such cases, allowing developers to cache a pre-generated SDK can drastically improve wakeup performance.
Key upcoming features include:
- SDK Caching: Developers can opt out of dynamic SDK generation and instead rely on a fixed SDK, improving cold start times.
- Regeneration Metadata: Server functions will display the timestamp of the last SDK generation and include a simple button to trigger regeneration when needed.
- Dependency Awareness: Future updates may enable the platform to identify which SDK functions are used by which server functions. This could offer precise guidance on when to regenerate or redeploy server functions due to SDK or code changes.
How to Try These Enhancements
To experiment with the latest improvements, use the native CLI deploy command:
npx poly function add –generateContexts <context-name> –no-types
Equivalent commands are coming for Python and Java SDKs.
Questions or Feedback?
Have questions about our roadmap or platform? We’d love to hear from you! Reach out to us at hello@polyapi.io, fill out a contact form, or book an intro meeting.